Doc-to-LoRA

Doc-to-Atom: Compositional parametric memory via semantically typed micro-LoRA adapters

Doc-to-Atom (Doc2Atom) proposes a framework that decomposes documents into semantically typed knowledge atoms, each compiled into an independent micro-LoRA adapter with a retrieval key. At inference, a lightweight query router assembles only relevant atoms into a query-specific adapter injected into a frozen base model, addressing the irrelevant-query interference and scalability problems of monolithic adapter approaches like Doc-to-LoRA. The system is trained end-to-end via multi-objective distillation and outperforms Doc-to-LoRA baselines on six QA benchmarks while reducing memory cost.