5arXiv cs.CL (Computation and Language)·
Disentangling solvability knowledge from verbalization in LLM hidden states
A new arXiv preprint probes LLM internal representations to separately decode 'solvability knowledge' and 'verbalization' as distinct linear directions in hidden states. The authors find that fabrication (hallucination of solutions to unsolvable problems) is primarily driven by shifts in verbalization representations rather than underlying knowledge. Prompting with unsolvability cues and activation steering can mechanistically shift verbalization to improve model abstention. The work advances mechanistic understanding of why LLMs confabulate on unsolvable math problems.
Related guides (2)
Related events (8)
Paper argues LLMs learn causal structure via difference-making logic, not Pearl/Rubin frameworks
A new arXiv preprint proposes that LLMs learn causal structure through 'variational induction' — a difference-making logic — rather than through the dominant formalisms of Judea Pearl's interventionist approach or the Neyman-Rubin potential outcomes framework. The author analyzes how this logic is realized during training and maps specific architectural features (token embeddings, self-attention) to their roles in this inductive process. The argument draws a parallel between LLM causal learning and the experimental method of systematically varying circumstances. This is a theoretical contribution to understanding how LLMs represent causal and world-model structure.
Revisiting LLM systematicity in negation understanding via in-context learning
A new arXiv preprint analyzes how well large language models handle negation from two angles: behavioral systematicity (whether models correctly recognize negation expressions and scope) and representational systematicity (whether function vectors can be reliably constructed from in-context examples). Results show LLMs partially succeed at negation cue recognition via in-context learning but struggle with scope recognition, with performance varying by output format. Function vectors can be composed for cue extraction but are harder to extract for scope recognition tasks.
Question-Answering as Hidden State Probing for Test-Time Reasoning Intervention
This paper proposes using question-asking as an inference-time intervention to surface information about an LLM's hidden state during chain-of-thought reasoning. The authors train a probe on a student model's hidden states before and after question generation, finding it predictive of final answer correctness even before the teacher responds—suggesting self-diagnosis during question generation carries meaningful signal. They frame question-asking as a sequential decision problem with a gating policy, but find a gap between detection and recovery: interventions are as likely to harm correct trajectories as to fix incorrect ones. The results have implications for the limits of LLM self-refinement under uncertainty.
Benchmarking study finds LLMs fail at counterintuitive probability problems despite strong standard performance
A new arXiv paper evaluates 8 state-of-the-art LLMs on discrete probability problems using two datasets: standard exercises (average accuracy 0.96) and counterintuitive exercises designed to trigger heuristic reasoning (average accuracy 0.59). The authors document token bias causing 20%+ performance drops when canonical problem formulations are disguised, and up to 34% degradation when misleading suggestions are embedded in prompts. The findings argue that current LLMs are not genuine probabilistic reasoners despite their success on advanced math benchmarks.
Survey chapter on LLM mechanisms, emergent capabilities, and cognition debates
A new arXiv preprint surveys current understanding of large language models, covering the Transformer architecture, emergent capabilities resembling human cognition (symbolic reasoning, theory of mind, deception), and explainability approaches from neuron activation analysis to circuit tracing. The chapter also engages the debate over whether LLMs genuinely understand or merely pattern-match, arguing against reductive anti-anthropomorphism while acknowledging human-LLM differences. It is framed as a book chapter synthesizing recent empirical findings and theoretical positions.
Theoretical framework for grounding LLM reasoning under incomplete knowledge graph evidence
A new arXiv preprint develops a formal theoretical framework for understanding how LLMs reason when guided by incomplete knowledge graphs. The authors introduce constructs including entity anchors, typed relation residuals, path energies, and support regions, and prove that under open-world incompleteness no hard rule can simultaneously reject all false unsupported trajectories while retaining all true-but-unobserved ones. Soft grounding is characterized as a KL-regularized deformation of the LLM prior, with hard conditioning as an infinite-penalty limit. The framework yields stability bounds under evidence perturbations and has implications for GraphRAG, KGQA, graph agents, constrained decoding, and faithful generation.
Study compares human and LLM active causal reasoning, finding LLMs less efficient but near human-level on conjunctive rules
A new arXiv paper investigates whether active exploration reduces the 'conjunctive handicap' in causal learning, using a blicket detector task with adult participants who could freely intervene to identify causal objects. Results show active exploration substantially improves human conjunctive causal reasoning, though conjunctive rules still require more tests than disjunctive ones. State-of-the-art LLMs approach human-level hypothesis inference accuracy but show less efficient exploration strategies and similar conjunctive-disjunctive performance gaps, raising questions about the nature of LLM causal reasoning.
Linear probes reveal LLMs encode remaining output length in hidden states
Researchers trained minimal linear probes on frozen hidden states of three open-weight 7-8B models and found that total response length is linearly decodable from the prompt's final hidden state before any output is generated. The probe directions transfer across natural-language and synthetic datasets, and per-position estimates shift upward when models retract and restart partial solutions. The authors interpret this as evidence that LLMs maintain a plan-like internal representation of remaining generation length, distinct from exact-counting, though causality is not established.