Entity · paper

Extending LLM Context via Associative Recurrent Memory

paperactiveprovisionalextending-llm-context-via-associative-recurrent-memory-63a66aee·1 events·first seen 39h ago

Aliases: Extending LLM Context via Associative Recurrent Memory

Co-occurring entities

Associative Recurrent Memory Transformer

More like this (12)

long-context LLMs Associative Recurrent Memory Transformer Continual LLM Upcycling: A Predictor-Gated Bank-Wise Sparsity Training Recipe for Dense-to-Sparse LLMs Co-LMLM: Continuous-Query Limited Memory Language Models Towards Root Memories: Benchmarking and Enhancing Implicit Logical Memory Retrieval for Personalized LLMs A sleep-like consolidation mechanism for LLMs Attention Amnesia in Hybrid LLMs: When CoT Fine-Tuning Breaks Long-Range Recall, and How to Fix It AIR: Adaptive Interleaved Reasoning with Code in MLLMs LLM inference Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs Latent Context Language Models ExpRL: Exploratory RL for LLM Mid-Training

Recent events (1)

6arXiv · cs.CL·39h ago·source ↗

Associative Recurrent Memory Transformer enables long-context LLMs with constant memory scaling

Researchers propose using the Associative Recurrent Memory Transformer (ARMT) as a practical method for extending LLM context length beyond original limits while achieving constant memory scaling instead of linear. The training recipe combines continued pre-training, synthetic long-context data, curriculum learning, and selective layer integration of associative memory. Experiments show ARMT-augmented models generalize to out-of-distribution context lengths and require 30% fewer FLOPs while preserving in-window baseline performance.

Long Context Evolution Inference Economics Extending LLM Context via Associative Recurrent Memory Associative Recurrent Memory Transformer