Entity · technique

Knowledge-Less Language Models

techniqueactiveprovisionalknowledge-less-language-models-6096d535·1 events·first seen 5d ago

Aliases: Knowledge-Less Language Models

Co-occurring entities

Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling

More like this (12)

Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling Latent Context Language Models Language Models are Few-Shot Learners Random Language Model Reasoning Language Models Tapered Language Models On the Limits of Prompt-Conditioned Language Models as General-Purpose Learners Transformer Language Models Language Modeling Loss Diffusion Language Models unsupervised language modeling Adaptive Multi-Resolution Procedural Knowledge Compression for Large Language Models

Recent events (1)

6arXiv · cs.CL·5d ago·source ↗

Knowledge-Less Language Models (KLLMs) suppress parametric recall via entity anonymization during pretraining

Researchers introduce Knowledge-Less Language Models (KLLMs), pretrained on corpora with anonymized named entities to systematically reduce parametric factual recall and push models toward evidence-grounded reasoning. Across multiple model scales, KLLMs outperform standard baselines on contextual QA, fact verification, and hallucination detection, with 20-25% relative gains in retrieval-grounded settings with imperfect evidence. The models also show improved calibration (ECE, Brier score, AUROC) and more reliable abstention behavior. The work suggests pretraining-time control over knowledge acquisition can serve as a complementary foundation for RAG and tool-augmented systems.

Evaluation and Benchmarking Alignment and RLHF Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling Knowledge-Less Language Models