MemSFT

MemSFT: Plug-and-play parametric memory module mitigates alignment tax in domain-specialized LLMs

Researchers propose MemSFT, a method that decouples domain specialization from backbone parameter updates by training a plug-and-play parametric memory module to imitate a non-parametric retriever over domain data. A learned router dynamically fuses the memory and backbone output distributions at each decoding step, allowing selective invocation of domain expertise. Evaluated across biology, geoscience, and law on models from Qwen3-8B to Qwen3-235B-A22B, MemSFT consistently improves domain performance with negligible general-task degradation, whereas full SFT causes severe catastrophic forgetting. The memory module is reusable across LLM sizes, offering a practical path to modular domain specialization.