Sundial: A Family of Highly Capable Time Series Foundation Models
원문 초록
We introduce Sundial, a family of native, flexible, and scalable time series foundation models. To predict the next-patch's distribution, we propose a TimeFlow Loss based on flow-matching, which facilitates native pre-training of Transformers on time series without discrete tokenization. Conditioned on arbitrary-length time series, our model is pre-trained without specifying any prior distribution and can generate multiple probable predictions, achieving flexibility in representation learning beyond using parametric densities. Towards time series foundation models, we leverage minimal but crucial adaptations of Transformers and curate TimeBench with 1 trillion time points, comprising mostly real-world datasets and synthetic data. By mitigating mode collapse through TimeFlow Loss, we pre-train a family of Sundial models on TimeBench, which exhibit unprecedented model capacity and generalization performance on zero-shot forecasting. In addition to presenting good scaling behavior, Sundial achieves new state-of-the-art on both point forecasting and probabilistic forecasting benchmarks. We believe that Sundial's pioneering generative paradigm will facilitate a wide variety of forecasting scenarios.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
SEED: A Structural Encoder for Embedding-Driven Decoding in Time Series Prediction with LLMs
Unsupervised Anomaly Detection in Energy Time Series Data Using Variational Recurrent Autoencoders with Attention
Dual Dimensions Geometric Representation Learning Based Document Dewarping
Towards Interpretable Time Series Foundation Models
MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
Decomposing the Time Series Forecasting Pipeline: A Modular Approach for Time Series Representation, Information Extraction, and Projection
토론
이 논문과 연결된 포럼 글이 아직 없습니다.