Entity · paper

Localized Adaptation Reveals Distinct Learning Signatures in Transformers

paperactiveprovisionallocalized-adaptation-reveals-distinct-learning-signatures-in-transformers-c56216ff·1 events·first seen 2d ago

Aliases: Localized Adaptation Reveals Distinct Learning Signatures in Transformers

Co-occurring entities

LoRA

More like this (12)

Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks Dynamic Short Convolutions Improve Transformers First-Token Broadcasters: Mechanistic Origins of Language Identity and Distributed Robustness in Transformers permutation-invariant transformers Associative Recurrent Memory Transformer Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers Automated Compliance Mapping in Cloud Security with Domain-Adapted Sentence Transformers Sparse Transformer Listening with Attention: Entropy-Guided Explainability for Transformer-Based Audio Models Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers Alternating Token-Weighted Unlearning

Recent events (1)

6arXiv · cs.CL·2d ago·source ↗

Localized LoRA adaptation reveals distinct learning geometries across transformer layers and task types

A new arXiv preprint introduces a controlled benchmark spanning five learning objectives (lexical binding, factual association, behavioral policy, causal mapping, procedural reasoning) to study how adaptation site shapes what transformers learn, how well it generalizes, and how selectively it applies. The authors define each objective's 'adaptation geometry' as its profile under full-stack versus early-, middle-, or late-layer LoRA. Key findings include that factual association favors late-layer adaptation, behavioral learning separates late-layer acquisition from middle-layer gating, and causal/procedural transfer benefits most from middle or full-stack updates. Results replicate across five model families under parameter-matched controls, establishing adaptation site as a meaningful design variable.

Evaluation and Benchmarking Alignment and RLHF Localized Adaptation Reveals Distinct Learning Signatures in Transformers LoRA