model

xLSTM

modelactiveprovisionalxlstm-0d6d068b·1 events·first seen 6d ago

Aliases: xLSTM

Co-occurring entities

Mamba On Subquadratic Architectures: From Applications to Principles Gated DeltaNet-2

More like this (12)

mlx-lm XLM-R large language models PyTorch/XLA long-context LLMs Multimodal Large Language Models Sparse Transformer TensorRT-LLM CLP: Collocation-Length Prediction for Zero-Loss Adaptive Multi-Token Inference large language model agents A sleep-like consolidation mechanism for LLMs omlx

Recent events (1)

5arXiv · cs.LG·6d ago·source ↗

Comparative study finds xLSTM outperforms Mamba-2 and Gated DeltaNet on complex sequence tasks

A new arXiv paper compares three subquadratic sequence modeling architectures — xLSTM, Mamba-2, and Gated DeltaNet — across code model pre-training, LLM distillation, and time-series foundation model pre-training. xLSTM consistently delivers the strongest performance, which the authors attribute to more flexible and stable memory correction via its gating scheme. The paper provides a unified formulation and analysis of state tracking and memory dynamics across the three architectures, with corroborating results on synthetic length-generalization tasks.

Training Infrastructure Evaluation and Benchmarking Mamba On Subquadratic Architectures: From Applications to Principles Gated DeltaNet-2 +1 more