Doc-to-Atom: Compositional parametric memory via semantically typed micro-LoRA adapters
Doc-to-Atom (Doc2Atom) proposes a framework that decomposes documents into semantically typed knowledge atoms, each compiled into an independent micro-LoRA adapter with a retrieval key. At inference, a lightweight query router assembles only relevant atoms into a query-specific adapter injected into a frozen base model, addressing the irrelevant-query interference and scalability problems of monolithic adapter approaches like Doc-to-LoRA. The system is trained end-to-end via multi-objective distillation and outperforms Doc-to-LoRA baselines on six QA benchmarks while reducing memory cost.
Related guides (3)
Related events (8)
DocTrace: Structure-Aware On-Demand Hypergraph Memory for Long-Document QA
Researchers introduce DocTrace, a multi-agent RAG framework for long-document question answering that uses query-triggered knowledge organization rather than costly query-agnostic preprocessing. The system combines a lightweight document structural tree index, on-demand hypergraph working memory, and a graph-structured experience memory that stores successful reasoning plans for reuse. Evaluated on four long-document QA datasets, DocTrace outperforms the strongest baseline (ComoRAG) by up to 8.85% F1 and 4.40% EM while reducing computational cost by 53.32%.
ProtoAda: Prototype-Guided Adaptive Adapter Expansion for Multimodal Continual Instruction Tuning
ProtoAda is a new framework for Multimodal Continual Instruction Tuning (MCIT) that addresses a key failure mode in sparse Mixture-of-LoRA-Experts architectures: image-text similarity routing is format-blind and incorrectly merges tasks with similar semantics but different output structures (e.g., coordinate prediction vs. VQA). The method introduces format-aware task prototypes to guide both routing and adapter expansion, then consolidates compatible updates geometrically to reuse and refine existing parameters. Experiments across multiple benchmarks show improved performance, particularly on tasks whose answer formats are vulnerable to corruption by sequential fine-tuning.
Parametric Memory Law for LoRA Finetuning: Quantifying LLM Memory Capacity
This paper introduces the Parametric Memory Law, a power-law relationship linking loss reduction to effective parameters and sequence length during LoRA-based LLM finetuning. The authors identify a phase transition at the token level where prediction probability p > 0.5 constitutes a sufficient condition for verbatim recall under greedy decoding. Building on these findings, they propose MemFT, a threshold-guided optimization strategy that dynamically reallocates training budget toward sub-threshold tokens, improving memory fidelity and efficiency.
Infini Memory: Topic-structured persistent memory architecture for long-term LLM agents
Researchers propose Infini Memory, a persistent memory architecture for LLM agents that organizes memory as topic-structured documents rather than isolated records or summaries. New observations are staged in a buffer and periodically consolidated, while retrieval uses iterative agentic tool calls instead of a single lookup step. The system achieves 64.7% on MemoryAgentBench, with ablations showing complementary gains from topic-structured maintenance and iterative evidence inspection.
MemOS: Self-Evolving Memory OS for LLM Agents with Hybrid Retrieval and Token Savings
MemOS is an open-source TypeScript project providing a memory operating system layer for LLM and AI agents, featuring ultra-persistent memory, hybrid retrieval, and cross-task skill reuse. The project claims 35.24% token savings through its memory management approach. It has accumulated 9,329 GitHub stars with moderate daily momentum (+67). The system targets agent memory persistence and efficiency as a foundational infrastructure component.
Code2LoRA: Hypernetwork generates repository-specific LoRA adapters for code models with zero token overhead
Code2LoRA is a hypernetwork framework that generates repository-specific LoRA adapters for code language models, eliminating the inference-time token overhead of RAG or long-context injection. It supports both static repository snapshots and evolving codebases via a GRU-backed adapter updated per code diff. The authors introduce RepoPeftBench, a new benchmark of 604 Python repositories with static and evolution tracks, on which Code2LoRA-Static matches per-repository LoRA fine-tuning upper bounds and Code2LoRA-Evo outperforms a shared LoRA by 5.2 percentage points.
Loong: A Human-Like Long Document Translation Agent with Observe-and-Act Adaptive Context Selection
Loong is a long document translation agent that uses a 3E memory module (Essence-Exemplar-Entity) to store structured historical context, replacing passive full-context attention with RL-optimized adaptive context selection. The agent learns its context retrieval policy via reinforcement learning on self-sampled reasoning trajectories. Evaluations show average gains of up to 13.0 points across three metrics in English↔Chinese, German, and French translation directions, with strong generalization and robustness to noise in ultra-long documents.
AuRA: Distilling audio understanding into LLMs via LoRA adaptation
AuRA is a new method for integrating speech understanding into LLMs by distilling audio encoding capability directly into LoRA-adapted model weights, bypassing cascaded ASR-LLM pipelines. A lightweight audio embedding layer feeds speech to both an ASR encoder (teacher) and a LoRA-adapted LLM (student), with layer-wise distillation aligning hidden states. The approach claims to outperform cascaded systems, bridge-based adaptation baselines, and large-scale multimodal models on multiple speech-language benchmarks while enabling parallel end-to-end inference without large-scale multimodal training.