HAR-former: Hybrid Transformer with an Adaptive Time-Frequency Representation Matrix for Long-Term Series Forecasting

Abstract

Time series forecasting is crucial across various fields such as economics, energy, transportation planning, and weather prediction. Nevertheless, accurately modeling real-world systems is challenging due to their inherent complexity and non-stationarity. Traditional methods, which often depend on high-dimensional embeddings, can obscure multivariate relationships and struggle with performance limitations, especially when handling complex temporal patterns. To address these issues, we propose HAR-former, a Hybrid Transformer with an Adaptive Time-Frequency Representation Matrix, which combines the strengths of Multi-Layer Perceptrons (MLPs) and Transformers to process trend and seasonal components, respectively. The HAR-former leverages a novel adaptive time-frequency representation matrix to bridge the gap between the time and frequency domains, allowing the model to capture both long-range dependencies and localized patterns. Extensive experimental evaluation on eight real-world benchmark datasets demonstrates that HAR-former outperforms existing state-of-the-art (SOTA) methods, establishing it as a robust solution for complex time series forecasting tasks.