DRAformer: Differentially Reconstructed Attention Transformer for Time-Series Forecasting
원문 초록
Time-series forecasting plays an important role in many real-world scenarios, such as equipment life cycle forecasting, weather forecasting, and traffic flow forecasting. It can be observed from recent research that a variety of transformer-based models have shown remarkable results in time-series forecasting. However, there are still some issues that limit the ability of transformer-based models on time-series forecasting tasks: (i) learning directly on raw data is susceptible to noise due to its complex and unstable feature representation; (ii) the self-attention mechanisms pay insufficient attention to changing features and temporal dependencies. In order to solve these two problems, we propose a transformer-based differentially reconstructed attention model DRAformer. Specifically, DRAformer has the following innovations: (i) learning against differenced sequences, which preserves clear and stable sequence features by differencing and highlights the changing properties of sequences; (ii) the reconstructed attention: integrated distance attention exhibits sequential distance through a learnable Gaussian kernel, distributed difference attention calculates distribution difference by mapping the difference sequence to the adaptive feature space, and the combination of the two effectively focuses on the sequences with prominent associations; (iii) the reconstructed decoder input, which extracts sequence features by integrating variation information and temporal correlations, thereby obtaining a more comprehensive sequence representation. Extensive experiments on four large-scale datasets demonstrate that DRAformer outperforms state-of-the-art baselines.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
DATE: Dual Attentive Tree-aware Embedding for Customs Fraud Detection
The Power of Architecture: Deep Dive into Transformer Architectures for Long-Term Time Series Forecasting
Temporal Convolutional Attention Neural Networks for Time Series Forecasting
Interpretable Multivariate Time Series Forecasting with Temporal Attention Convolutional Neural Networks
Robust Online Time Series Prediction with Recurrent Neural Networks
MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.