5arXiv cs.LG (Machine Learning)·
TiRex-2: xLSTM-based time series foundation model for multivariate and streaming forecasting
TiRex-2 is a recurrent xLSTM-based time series foundation model that extends the univariate TiRex to multivariate forecasting with past and future covariates, while supporting streaming inference at constant per-patch cost. The model uses a bidirectional time mixer and asymmetric grouped-attention variate mixer to handle future-known covariates without violating causality over target variables. A synthetic coupling pipeline enables scalable multivariate pretraining from univariate corpora. TiRex-2 claims state-of-the-art zero-shot performance on GIFT-Eval and fev-bench benchmarks with 38.4M–82.5M parameters depending on mode.
Related guides (2)
Related events (8)
Dual-TIRE: Streaming neural speech codec with time-invariant factorized representations
A new arXiv preprint investigates TiCodec's Time-Invariant Representation Extraction (TIRE) module, which separates time-varying speech content from time-invariant speaker and environment information in neural speech codecs. The authors introduce Dual-TIRE, a multi-level architecture exploiting complementarity across encoder layers to improve reconstruction quality and speaker similarity. They also demonstrate that streaming inference using 660ms processing blocks achieves minimal degradation, suggesting practical viability for low-latency speech generation pipelines.
PatchTSMixer in HuggingFace
Hugging Face introduces PatchTSMixer, a lightweight MLP-Mixer-based model for multivariate time-series forecasting, now available in the Transformers library. The model is designed for efficient patch-based mixing of temporal and channel information. This integration expands Hugging Face's time-series modeling capabilities alongside the previously added PatchTST model.
Multivariate Probabilistic Time Series Forecasting with Informer
A Hugging Face blog post introduces the Informer model for multivariate probabilistic time series forecasting. The post covers the architecture and usage of Informer, which uses a sparse attention mechanism (ProbSparse) to handle long sequences more efficiently than standard Transformers. It demonstrates how to use the model via the Hugging Face Transformers library for forecasting tasks.
Toto 2.0: Open-Weights Time Series Foundation Models Demonstrate Scaling Laws from 4M to 2.5B Parameters
Datadog releases Toto 2.0, a family of five open-weights time series forecasting models ranging from 4M to 2.5B parameters, demonstrating consistent forecast quality improvements with scale. The models achieve state-of-the-art results on three benchmarks: BOOM (observability), GIFT-Eval (general-purpose), and TIME (contamination-resistant). The release includes architectural details, a u-muP hyperparameter transfer pipeline, and all base checkpoints under Apache 2.0 license.
Probabilistic Time Series Forecasting with Transformers
This Hugging Face blog post introduces probabilistic time series forecasting using Transformer-based models available in the Hugging Face ecosystem. It covers the application of attention-based architectures to sequential prediction tasks with uncertainty quantification. The post serves as a tutorial and capability demonstration for time series modeling within the Transformers library.
Test-Time Training End-to-End (TTT-E2E) Retrains Model Weights to Handle Long Inputs
Researchers from Astera Institute, Nvidia, Stanford, UC Berkeley, and UC San Diego introduced TTT-E2E, a method that compresses long context into transformer weights by training the model during inference via meta-learning. The approach uses sliding-window attention restricted to 8,000 tokens and updates only the fully connected layers of the last quarter of the network on each 1,000-token chunk at inference time, keeping per-token generation latency roughly constant as context scales to 128,000 tokens. TTT-E2E slightly outperforms vanilla transformers on next-token prediction loss across long contexts and matches efficient architectures like Mamba 2 and Gated DeltaNet on inference speed, but fails dramatically on Needle-in-a-Haystack retrieval beyond 8,000 tokens and incurs substantially higher training latency. The work reframes long-context handling as a training-inference trade-off rather than an architectural design problem.
Adaptive asymmetric token compression accelerates time series language models up to 7.68×
A new arXiv preprint proposes an adaptive token budgeting framework for time series (TS) language models that compresses TS tokens using frequency-domain structure and progressively prunes prompt tokens across model layers. The authors demonstrate up to 7.68× inference acceleration with performance improvements in 78% of evaluated settings across forecasting, classification, imputation, and anomaly detection tasks. The work is motivated by the observation that TS tokens have uneven spectral contributions and prompt-token influence attenuates with model depth, making uniform token processing wasteful.
Patch Time Series Transformer in Hugging Face
Hugging Face has integrated PatchTST, a patch-based Transformer architecture for time series forecasting, into its ecosystem. PatchTST applies the patching concept from vision transformers to time series data, dividing sequences into subseries-level patches as input tokens. The blog post covers usage, fine-tuning, and zero-shot transfer capabilities of the model within the Hugging Face Transformers library.