4arXiv cs.CL (Computation and Language)·
Dual-TIRE: Streaming neural speech codec with time-invariant factorized representations
A new arXiv preprint investigates TiCodec's Time-Invariant Representation Extraction (TIRE) module, which separates time-varying speech content from time-invariant speaker and environment information in neural speech codecs. The authors introduce Dual-TIRE, a multi-level architecture exploiting complementarity across encoder layers to improve reconstruction quality and speaker similarity. They also demonstrate that streaming inference using 660ms processing blocks achieves minimal degradation, suggesting practical viability for low-latency speech generation pipelines.
Related guides (1)
Related events (8)
TiRex-2: xLSTM-based time series foundation model for multivariate and streaming forecasting
TiRex-2 is a recurrent xLSTM-based time series foundation model that extends the univariate TiRex to multivariate forecasting with past and future covariates, while supporting streaming inference at constant per-patch cost. The model uses a bidirectional time mixer and asymmetric grouped-attention variate mixer to handle future-known covariates without violating causality over target variables. A synthetic coupling pipeline enables scalable multivariate pretraining from univariate corpora. TiRex-2 claims state-of-the-art zero-shot performance on GIFT-Eval and fev-bench benchmarks with 38.4M–82.5M parameters depending on mode.
TEVI: Sparse autoencoders for text-conditioned editing of CLIP image embeddings to improve vision-language alignment
TEVI is a framework that uses sparse autoencoders to disentangle CLIP image embeddings and a learned masking module to selectively reconstruct embeddings conditioned on a given caption, addressing the information imbalance between images and their captions. The approach improves image-text retrieval on both coarse-grained benchmarks (MS COCO, Flickr) and fine-grained long-caption benchmarks (IIW, DOCCI), with larger gains on richer captions. The work also shows improved robustness on the RoCOCO benchmark.
Controlled ablation reveals training artifact behind low frame rate degradation in neural audio codecs
A new arXiv preprint investigates why neural audio codecs degrade sharply at low frame rates (≤6.25 Hz), a property relevant to autoregressive speech synthesis where generation cost scales with sequence length. The authors reproduce a previously reported quality cliff at 6.25 Hz and show it stems from a suboptimal training configuration—fixed clip duration starves the decoder of inter-token context at low frame rates—rather than fundamental phonemic or codebook limits. After correcting the training setup, word error rate degrades smoothly down to 1.6 Hz, suggesting low frame rate codecs are more practically accessible than prior work implied.
Adaptive asymmetric token compression accelerates time series language models up to 7.68×
A new arXiv preprint proposes an adaptive token budgeting framework for time series (TS) language models that compresses TS tokens using frequency-domain structure and progressively prunes prompt tokens across model layers. The authors demonstrate up to 7.68× inference acceleration with performance improvements in 78% of evaluated settings across forecasting, classification, imputation, and anomaly detection tasks. The work is motivated by the observation that TS tokens have uneven spectral contributions and prompt-token influence attenuates with model depth, making uniform token processing wasteful.
REDDIT framework corrects timestamp drift in autoregressive ASR without catastrophic forgetting
A new arXiv paper introduces REDDIT (Replay-based Distribution EDITing), a two-stage post-training framework that corrects timestamp drift in autoregressive ASR systems like Whisper across long non-speech spans. The method updates only 1.6% of model parameters and constructs correction supervision without human annotations, using VAD-trimmed speech with inserted non-speech gaps. On Whisper-tiny, long-gap mIoU improves from 38.7% to 95.0% and out-of-domain alignment error drops from 2752 ms to 223 ms, while preserving transcription quality that ordinary SFT decoder tuning catastrophically degrades.
C-DIC: Context-Driven Incremental Compression for efficient long-horizon multi-turn dialogue
A new arXiv preprint introduces Context-Driven Incremental Compression (C-DIC), a method for managing growing dialogue history in conversational agents by treating conversations as interleaved contextual threads with revisable per-thread compression states stored in a compact dialogue memory. A retrieve-revise-write-back loop shares information across turns and updates stale memories, while truncated backpropagation-through-time (TBPTT) is adapted to learn cross-turn dependencies. Experiments on long-form dialogue benchmarks show stable inference latency and perplexity over hundreds of turns, addressing compounding errors seen in existing context compressors.
LeVo 2: Hybrid LLM-Diffusion framework for stable full-length song generation with hierarchical modeling
LeVo 2 is a new hybrid LLM-Diffusion system for controllable full-length song generation that addresses the coherence-vs-acoustics trade-off through hierarchical token prediction: a language model handles semantic planning via mixed tokens, then predicts vocal and accompaniment tracks in parallel, while a diffusion-based codec reconstructs waveforms. A key contribution is an aesthetics-guided progressive post-training schedule combining SFT, offline DPO, and semi-online DPO to separately optimize quality, controllability, and musicality. Expert listening tests show LeVo 2 outperforms open-source baselines across six subjective dimensions and approaches leading commercial systems on several metrics.
RABBiT: Foundation model for zero-shot and few-shot prediction of speech-elicited fMRI brain responses
Researchers introduce RABBiT, a compact audio-to-fMRI encoder foundation model designed to predict brain responses to natural speech with zero or few participant-specific data. Evaluated on 324 participants across multiple unseen fMRI datasets, RABBiT outperforms the current state-of-the-art fMRI foundation model and group-average baselines in auditory and language-selective brain regions. With only 10 minutes of participant data, parameter-efficient fine-tuning further improves performance substantially over per-participant linear models. Key innovations include learned region-specific attention and a decomposition of brain responses into shared and subject-specific components.