SmolLM2-360M

ACROS: Inducing Sense Representations as Interfaces into Frozen Pretrained LMs

ACROS introduces a method to add explicit sense representations (per-token meaning decompositions) to frozen pretrained decoder language models via a gated residual addition, without requiring sense-aware pretraining. On SmolLM2-360M, the induced sense pathway supports zero-shot word-sense disambiguation (64.95 F1 on Raganato ALL), lexical steering across 5,161 CoInCo cases, and cross-lingual adaptation to four languages with near-perfect retrieval accuracy. The approach preserves base LM quality while enabling three distinct downstream uses from the same induced variables, positioning sense representations as a post-hoc modular interface for ordinary pretrained models.