on-subquadratic-architectures-from-applications-to-principles-1d849fe9·1 events·first seen Aliases: On Subquadratic Architectures: From Applications to Principles
A new arXiv paper compares three subquadratic sequence modeling architectures — xLSTM, Mamba-2, and Gated DeltaNet — across code model pre-training, LLM distillation, and time-series foundation model pre-training. xLSTM consistently delivers the strongest performance, which the authors attribute to more flexible and stable memory correction via its gating scheme. The paper provides a unified formulation and analysis of state tracking and memory dynamics across the three architectures, with corroborating results on synthetic length-generalization tasks.