PaSTS: Parameter-affined Seasonal-Trend Synthesis for Multi-dimensional Long-Term Time Series Forecasting within LLM

Abstract

Large Language Models (LLMs) have demonstrated remarkable performance across various domains, showcasing significant potential for long-term time series forecasting (LTSF), and consequently attracting substantial research interest. In LTSF, temporal decomposition has been widely adopted in existing models, including both Transformer-based and linear models, to enhance predictive capabilities. However, our experiments indicate that a simplistic integration of these decomposition methods into LLMs can lead to overfitting, even though they are effective in traditional models. In this paper, we propose PaSTS, a novel framework designed to integrate decomposition methods into LLMs through a specialized temporal synthesis layer, thereby improving predictive accuracy and mitigating overfitting of LLMs in LTSF tasks. Empirical evaluation of our framework provides evidence supporting the effective integration of LLMs with temporal decomposition techniques. Furthermore, applying our synthesis method to the decomposed series in several traditional models that employ seasonal-trend decomposition demonstrates its adaptability.