4arXiv cs.CL (Computation and Language)·
Surprisal theory in psycholinguistics is tautological without rational grounding, paper argues
A new arXiv preprint argues that surprisal theory — the claim that human linguistic processing difficulty is an affine function of surprisal under some language model — is unfalsifiable without additional constraints, because for any difficulty measure a language model can be constructed to match it. The paper identifies that two decades of psycholinguistic work implicitly assumed the relevant language model was the corpus-generating distribution, an assumption recent empirical work has undermined by showing better corpus models can be worse predictors of processing difficulty. The author concludes that breaking the tautology requires deriving the relevant language model from a rationalist, non-empirically motivated model of the comprehender (e.g., based on memory constraints or processing goals).
Related guides (1)
Related events (8)
Preprint argues LLM-based surprisal metrics obscure representational commitments in psycholinguistics
A new arXiv preprint challenges the common practice in computational psycholinguistics of treating LLM-derived surprisal values as theory-neutral, representation-agnostic measures. The authors argue that algorithm choice and model architecture significantly affect computed language model probabilities, meaning LLM surprisal is not interchangeable across models. Three analyses demonstrate that using black-box LLMs uncritically conflates computational-level and algorithmic-level commitments, undermining the theoretical claims of Surprisal Theory.
Energy-based transformers as unified predictors of reading difficulty in computational psycholinguistics
A new arXiv preprint introduces energy-based transformer measures as predictors of human reading difficulty, evaluated across three reading-time corpora (Natural Stories, UCL eye-tracking, UCL self-paced reading). The energy measure outperforms surprisal alone and appears to subsume both surprisal and attention entropy effects, suggesting it could serve as a single unified predictor. The work connects transformer language models to Hopfield networks and dense associative memory literature, marking the first application of energy-based transformer measures in computational psycholinguistics.
Why Language Models Hallucinate
OpenAI published research explaining the mechanisms behind language model hallucination. The work connects improved evaluation methods to enhanced AI reliability, honesty, and safety. The body is sparse on technical detail, but the framing positions this as foundational research relevant to alignment and deployment trust.
Quantifying Cross-Linguistic Effects of Syncretism on Agreement Attraction Using LLM Processing Proxies
This paper investigates why morphological syncretism amplifies agreement attraction errors in some languages (English, German, Russian) but not others (Turkish, Armenian), a pattern lacking a principled account. The authors use surprisal and attention entropy derived from large language models as proxies for human sentence processing across four languages. LLM-derived measures successfully replicate behavioral findings in English and German, align with Turkish null results, and partially capture Russian patterns. The work demonstrates LLMs as tools for cross-linguistic psycholinguistic investigation.
Diagnostic framework decomposes LLM difficulty on historical Italian and Russian texts
A new arXiv preprint proposes a four-dimensional framework for measuring LLM difficulty on historical language: tokenization cost, surprisal, semantic robustness, and context sensitivity. Evaluated on 17th-century Italian, 19th-century Italian, and 18th-century Russian texts, the study finds that tokenization penalties (25-30% inflation) are similar across languages but predictive difficulty diverges sharply—early modern Italian is 2.4x more surprising than modern Italian while Russian shows only modest increase. Crucially, embedding similarity remains high (>0.85) even when generation is unstable, and a simple temporal context prompt reduces historical surprisal by ~60%. The findings have practical implications for deploying LLMs in digital library and historical document workflows.
PAC-Bayes analysis establishes formal expressivity and alignment floors for prompt-conditioned LLMs
A new arXiv preprint models user-LLM interaction as a bilevel cheap-talk game and derives PAC-Bayes bounds showing two irreducible limitations: an 'expressivity floor' where language's finite channel capacity makes distinct tasks indistinguishable, and an 'objective-misalignment floor' where alignment constraints prevent reaching user-ideal outputs. The authors prove that prompt-conditioned LLMs cannot be universal problem solvers, as correct behavior on certain task families is provably unattainable even with infinite data, optimal training, or model scaling. The work suggests multimodal inputs and external memory as potential mitigations by increasing task-relevant information bandwidth.
Neural Collapse Is Forbidden: Information Floors in Language Model Representations
A new arXiv preprint argues that within-class variance in language model representations is not incomplete neural collapse but rather allocated information storage obeying a measurable law. Across 14 models spanning a 100x parameter range, macro-category structure accounts for only 4-12% of representational variance while within-token context carries 79-91%. The authors prove a theoretical floor on within-category dispersion proportional to the conditional mutual information I(token; context | category), and show this law holds across models, partitions, and over pretraining dynamics, including voiding a family of simplex ETF claims.
Survey chapter on LLM mechanisms, emergent capabilities, and cognition debates
A new arXiv preprint surveys current understanding of large language models, covering the Transformer architecture, emergent capabilities resembling human cognition (symbolic reasoning, theory of mind, deception), and explainability approaches from neuron activation analysis to circuit tracing. The chapter also engages the debate over whether LLMs genuinely understand or merely pattern-match, arguing against reductive anti-anthropomorphism while acknowledging human-LLM differences. It is framed as a book chapter synthesizing recent empirical findings and theoretical positions.