Deep autoregressive models with spectral attention Articles uri icon

publication date

  • January 2023

start page

  • 1

end page

  • 12

issue

  • 109014

volume

  • 133

International Standard Serial Number (ISSN)

  • 0031-3203

abstract

  • Time series forecasting is an important problem across many domains, playing a crucial role in multiple real-world applications. In this paper, we propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module, which merges global and local frequency domain information in the model's embedded space. By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns. Two spectral attention models, global and local to the time series, integrate this information within the forecast and perform spectral filtering to remove time series's noise. The proposed architecture has a number of useful properties: it can be effectively incorporated into well-known forecast architectures, requiring a low number of parameters and producing explainable results that improve forecasting accuracy. We test the Spectral Attention Autoregressive Model (SAAM) on several well-known forecast datasets, consistently demonstrating that our model compares favorably to state-of-the-art approaches.

subjects

  • Telecommunications

keywords

  • attention models; deep learning; filtering; global-local contexts; signal processing; spectral domain attention; time series forecasting