Entity · paper

Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning

paperactiveprovisional

towards-mechanistically-understanding-why-memorized-knowledge-fails-to-generalize-in-large-language-model-finetuning-5c7d0cfa

·1 events·first seen 3d ago

Aliases: Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning

Co-occurring entities

self-patching

More like this (12)

Language Model Finetuning Understanding Large Language Models Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustness Adaptive Multi-Resolution Procedural Knowledge Compression for Large Language Models Unintended Effects of Geographic Conditioning in Large Language Models Attention Amnesia in Hybrid LLMs: When CoT Fine-Tuning Breaks Long-Range Recall, and How to Fix It On the Limits of Prompt-Conditioned Language Models as General-Purpose Learners Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact Future Confidence Distillation in Large Language Models Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs Single and Multi Truth Data Fusion using Large Language Models

Recent events (1)

6arXiv · cs.CL·3d ago·source ↗

Self-patching technique reveals why fine-tuned LLMs fail to generalize memorized knowledge

A new arXiv preprint formalizes the 'Knowing–Using Gap' in LLM fine-tuning: models can memorize injected facts yet fail to apply them in downstream reasoning. The authors introduce a novel intervention technique called self-patching to monitor internal activation dynamics, finding evidence for a knowledge-circuit misalignment hypothesis where memorized representations are not routed to computation-effective layers. A heuristic strategy derived from this diagnostic recovers 58–75% of the oracle headroom in generalization failure cases across multiple domains.

Evaluation and Benchmarking Alignment and RLHF Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning self-patching