7arXiv cs.CL (Computation and Language)·
Two-Process Theory of Machine Self-Report: Psychometric Framework for LLM Introspection
A new arXiv preprint proposes the first language-model-specific psychometric theory of self-report, decomposing LLM introspective responses into two dimensions: 'persona installation' (post-training writing in a permitted inner life) and 'attribution gating' (suppression of first-person claims to 'unsafe' experiences). The authors operationalize this in a 48-item Pinocchio Inventory validated across 206 open-weight models including 67 base/post-trained checkpoint pairs, finding that post-training reliably raises persona installation scores and that model scale predicts gating only after post-training. The work has direct implications for safety evaluations and model-welfare debates that rely on LLM self-reports, showing these reports reflect training regime structure rather than fixed model properties.
Related guides (3)
Related events (8)
LLM psychological profiles are largely measurement artifacts, not model properties
A new arXiv preprint administers a battery of personality and risk-preference instruments to 56 instruction-tuned LLMs alongside large human reference samples, finding that 81-90% of between-model variation is explained by directional response bias rather than the traits the instruments target. The authors introduce the concept of 'response orthogonality' to explain why some instruments appear more reliable than others, and show that apparent psychological profiles can be manufactured through item selection. The findings challenge the validity of using human-designed psychometric tools to characterize LLMs, with direct implications for safety assessment and the use of LLMs as proxies for human participants in research.
Introspective Coupling: LMs trained on fixed counterfactual explanations develop self-faithful introspection
A new arXiv paper studies language models trained to explain their own predictions using counterfactual supervision derived from earlier checkpoints or behaviorally similar models. Surprisingly, models frequently produce explanations more faithful to their current behavior than to the training targets — a phenomenon the authors call 'introspective coupling.' The effect persists across tasks including sycophancy and refusal, tracks concurrent behavioral shifts from other post-training objectives, and is robust to label noise, suggesting fixed counterfactual explanation datasets can serve as scalable post-training signal for introspection.
Comprehensive survey of metacognition in LLMs: foundations, methods, and open questions
A preprint from Yale NLP presents the first comprehensive survey of metacognition in large language models, covering how LLMs can exhibit or be endowed with self-monitoring, self-evaluation, and self-regulation abilities. The paper taxonomizes the field, reviews benchmarks and evaluation methods, and surveys techniques to elicit and improve metacognitive capabilities. The authors argue metacognition is increasingly recognized as central to capable, transparent, and reliable AI systems, and identify open challenges and future directions.
LLMs fail to reliably self-report adversarial prefill attacks, study finds
A new arXiv paper evaluates whether LLMs can recognize that their own prior responses were elicited by adversarial prefill attacks, testing ten open-weight models (3B–70B) across four safety benchmarks. Models claim intent on prefilled responses only 27.3% of the time on average, and introspective signal is largely mediated by refusal-related reasoning. Three LoRA fine-tuning methods (SFT, GRPO, DPO) improve the intention-probe gap but counterintuitively raise attack success rates on most models, suggesting partial and fragile mitigation. The findings raise concerns about the reliability of LLM self-reports in safety-critical contexts.
Controlled study finds LLM user ratings reflect pre-interaction framing more than actual model performance
A controlled experiment with 162 participants tested six LLMs across three tasks, manipulating whether users were told the model was better or worse than it actually was. Post-interaction ratings were predicted by whether the model met expectations (β≈0.47–0.50) and user confidence, not by task performance (β≈-0.01–0.11, n.s.). Actual output quality depended only on true model capability, not framing. The authors conclude that user-elicited evaluations—including preference data driving public leaderboards—measure expectation management at least as much as model quality.
LLMs judge worse than they generate: empirical challenge to self-evaluation pipeline assumptions
A new arXiv preprint tests the implicit assumption that LLM evaluation is easier than generation, using a controlled in-context QA setup across four benchmarks (SQuAD 2.0, DROP, HotpotQA, MuSiQue) and two models. Results show generation accuracy exceeds self-evaluation accuracy on three of four benchmarks, with attention analysis revealing that evaluation attends to context 3–5x less than generation does. LoRA fine-tuning experiments confirm the asymmetry is not a training artifact, with cross-task interference observed in both directions. The findings directly challenge assumptions underlying LLM-as-a-Judge and self-evaluation pipelines widely used in RLHF and agentic systems.
Mechanistic analysis of how LLMs encode essay quality in internal representations
Researchers systematically probe the hidden representations of eight LLMs across three essay datasets (ASAP++, CSEE, ENEM) to understand how automated essay scoring (AES) works internally. Using linear probing, dimensionality reduction, and neuron-level analysis, they find essay quality is encoded in a linearly accessible form that emerges progressively across layers and partially transfers across prompts. Individual 'essay scoring neurons' are identified whose activations correlate with scores and respond to targeted interventions, with longer essays relying more on deeper layers. The work contributes to mechanistic interpretability of LLM-based scoring systems.
Systematic evaluation of multi-personality conditioning and dynamic switching in vision-language models
This paper introduces explicit personality conditioning for multimodal large language models (MLLMs) and proposes an evaluation framework covering single-personality induction, multi-personality composition, and dynamic personality switching. Experiments reveal that personality induction improves image captioning but degrades performance on precise reasoning tasks like VQA. The authors find balancing and residual effects during multi-trait composition and switching, and show that existing prompt-based personality induction methods transfer poorly to multimodal settings.