Entity · paper

Co-LMLM: Continuous-Query Limited Memory Language Models

paperactiveprovisionalco-lmlm-continuous-query-limited-memory-language-models-0d9a5c03·1 events·first seen 2d ago

Aliases: Co-LMLM: Continuous-Query Limited Memory Language Models

Co-occurring entities

GPT-4o mini Claude Sonnet 4.5 CO-LMLM SimpleQA FineWeb-Edu

More like this (12)

Multimodal Large Language Models CO-LMLM continuous diffusion language model Recursive Language Models (RLMs)Character-level Language Model Latent Context Language Models large language models Towards Root Memories: Benchmarking and Enhancing Implicit Logical Memory Retrieval for Personalized LLMs mRNA Language Model Self-Compacting Language Model Agents Random Language Model Which Models Are Our Models Built On? Auditing Invisible Dependencies in Modern LLMs

Recent events (1)

6arXiv · cs.LG·2d ago·source ↗

Co-LMLM: Continuous-query limited memory language models outperform vanilla LLMs on factual tasks at small scale

Researchers introduce CO-LMLM, a limited memory language model that externalizes factual knowledge to a knowledge base during pretraining and retrieves it at inference via continuous vector queries paired with human-readable text values. The approach removes prior restrictions to relational knowledge bases and Wikipedia-only data by introducing an annotation pipeline for arbitrary text. At 360M parameters, CO-LMLM achieves lower perplexity than models trained on 40x more data and SimpleQA factual performance comparable to GPT-4o mini and above Claude Sonnet 4.5, suggesting significant efficiency gains for factual grounding.

Evaluation and Benchmarking Open Weights Progress Co-LMLM: Continuous-Query Limited Memory Language Models GPT-4o mini Claude Sonnet 4.5 +4 more