Entity · paper

Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling

paperactiveprovisionalknowledgeless-language-models-suppressing-parametric-recall-for-evidence-grounded-language-modeling-e8349b8b·1 events·first seen 6d ago

Aliases: Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling

Co-occurring entities

Knowledge-Less Language Models

More like this (12)

Knowledge-Less Language Models Evaluation Awareness Is Not One Capability: Evidence from Open Language Models On the Limits of Prompt-Conditioned Language Models as General-Purpose Learners Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Models Language Modeling Loss Reasoning Language Models Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning encoder-only language models Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models Language-Critique Imitation Learning from Suboptimal Demonstrations Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustness Language Models are Few-Shot Learners

Recent events (1)

6arXiv · cs.CL·6d ago·source ↗

Knowledge-Less Language Models (KLLMs) suppress parametric recall via entity anonymization during pretraining

Researchers introduce Knowledge-Less Language Models (KLLMs), pretrained on corpora with anonymized named entities to systematically reduce parametric factual recall and push models toward evidence-grounded reasoning. Across multiple model scales, KLLMs outperform standard baselines on contextual QA, fact verification, and hallucination detection, with 20-25% relative gains in retrieval-grounded settings with imperfect evidence. The models also show improved calibration (ECE, Brier score, AUROC) and more reliable abstention behavior. The work suggests pretraining-time control over knowledge acquisition can serve as a complementary foundation for RAG and tool-augmented systems.

Evaluation and Benchmarking Alignment and RLHF Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling Knowledge-Less Language Models