paper

Words as Difference Makers: How Large Language Models Determine Causal Structure in Text

paperactiveprovisionalwords-as-difference-makers-how-large-language-models-determine-causal-structure-in-text-112a7850·1 events·first seen 45h ago

Aliases: Words as Difference Makers: How Large Language Models Determine Causal Structure in Text

Co-occurring entities

Judea Pearl self-attention

More like this (12)

Reasoning Language Models From Texts to Scores: Tracing the Emergence of Essay Quality Representations in Large Language Models Same Lesson, Different Story: Cross-Lingual Reconstruction of Cultural Narratives in Large Language Models Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact Automated reproducibility assessments in the social and behavioral sciences using large language models Unintended Effects of Geographic Conditioning in Large Language Models causal generative modeling Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustness Latent Context Language Models Contrastive-Difference CKA Reveals Concept-Specific Structural Alignment Across Language Model Architectures large language models The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models

Recent events (1)

5arXiv · cs.CL·45h ago·source ↗

Paper argues LLMs learn causal structure via difference-making logic, not Pearl/Rubin frameworks

A new arXiv preprint proposes that LLMs learn causal structure through 'variational induction' — a difference-making logic — rather than through the dominant formalisms of Judea Pearl's interventionist approach or the Neyman-Rubin potential outcomes framework. The author analyzes how this logic is realized during training and maps specific architectural features (token embeddings, self-attention) to their roles in this inductive process. The argument draws a parallel between LLM causal learning and the experimental method of systematically varying circumstances. This is a theoretical contribution to understanding how LLMs represent causal and world-model structure.

Alignment and RLHF Judea Pearl self-attention Words as Difference Makers: How Large Language Models Determine Causal Structure in Text