paper

Scaling limit of the Random Language Model

paperactiveprovisionalscaling-limit-of-the-random-language-model-2acd13bd·1 events·first seen 19h ago

Aliases: Scaling limit of the Random Language Model

Co-occurring entities

Random Energy Model Random Language Model

More like this (12)

Random Language Model Scaling Laws for Neural Language Models Language Generation in the Limit On the Limits of Prompt-Conditioned Language Models as General-Purpose Learners Space-Efficient Language Generation in the Limit Tapered Language Models The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models LESS: Mutual-Stability Sampling for Diffusion Language Models multi-turn language models 1B-scale language models Civil Court Simulation with Large Language Models Dango: A Strictly L1-Only Large Language Model for Studying Second Language Acquisition

Recent events (1)

4arXiv · cs.CL·19h ago·source ↗

Scaling limit theory of the Random Language Model reveals condensation transition and language statistics

A new arXiv preprint develops a quantitative theory of the Random Language Model (RLM), an ensemble of stochastic context-free grammars, in a scaling limit where grammar size and temperature are jointly tuned. The authors identify a condensation phase transition at a critical parameter value and derive explicit scaling laws for entropy, rule diversity, and related observables across distinct regimes. The work claims to resolve prior ambiguities about thermodynamic transitions in language models and offers a unified framework connecting generative grammar statistics to universal properties of natural language and LLM behavior.

Evaluation and Benchmarking Random Energy Model Random Language Model Scaling limit of the Random Language Model