5arXiv cs.CL (Computation and Language)·
Survey argues LLMs should be designed with explicit conceptual representations, not just probed for them
A new arXiv preprint proposes treating 'concepts' as a first-class design axis for LLMs, contrasting the current norm of recovering concept-like structure post-hoc (via probing or dictionary learning) with the alternative of deliberately engineering it into training objectives, architectures, or inference procedures. The authors map the design space along two dimensions: pipeline stage and whether structure is internally derived or externally grounded. Key findings include that inference-time approaches are underexplored, related ideas have developed in isolation across pipeline stages, and externally grounded methods span the full pipeline under inconsistent terminology.
Related guides (2)
Related events (8)
LLMs can extract and reuse experiential abstractions to improve reasoning, study finds
Researchers investigate whether LLMs can distill solution traces into reusable natural-language abstractions (strategies, reminders) and apply them to future problems, analogous to human experiential learning. Using the MATH training set, abstractions are extracted by either a stronger teacher model or the LLMs themselves into a retrievable library, then applied via inference-time retrieval or RL with abstraction-augmented prompts. Both modes improve performance on mathematical and logical reasoning benchmarks, with self-extracted abstractions matching teacher-extracted ones. The framework generalizes across datasets and models, suggesting a promising direction for continual self-improvement in LLMs.
Survey chapter on LLM mechanisms, emergent capabilities, and cognition debates
A new arXiv preprint surveys current understanding of large language models, covering the Transformer architecture, emergent capabilities resembling human cognition (symbolic reasoning, theory of mind, deception), and explainability approaches from neuron activation analysis to circuit tracing. The chapter also engages the debate over whether LLMs genuinely understand or merely pattern-match, arguing against reductive anti-anthropomorphism while acknowledging human-LLM differences. It is framed as a book chapter synthesizing recent empirical findings and theoretical positions.
Paper argues LLMs learn causal structure via difference-making logic, not Pearl/Rubin frameworks
A new arXiv preprint proposes that LLMs learn causal structure through 'variational induction' — a difference-making logic — rather than through the dominant formalisms of Judea Pearl's interventionist approach or the Neyman-Rubin potential outcomes framework. The author analyzes how this logic is realized during training and maps specific architectural features (token embeddings, self-attention) to their roles in this inductive process. The argument draws a parallel between LLM causal learning and the experimental method of systematically varying circumstances. This is a theoretical contribution to understanding how LLMs represent causal and world-model structure.
Study finds LLM-generated research ideas cluster around synthesis and bridging, diverging from human distribution
A new arXiv paper introduces a large-scale evaluation framework for comparing LLM-generated research ideas against human-authored ones, using reverse-engineered prior-work sets as prompts. The authors develop a two-axis taxonomy of research taste (opportunity pattern and research paradigm) and find a consistent distributional gap: LLMs over-index on bridge-like opportunities and synthesis methods, while human researchers spread more broadly across framing and contribution types. The result suggests current LLMs produce reasonable but systematically narrower and shifted ideation relative to human researchers.
Theoretical framework proposes native meta-architecture to replace application-layer LLM cognitive simulation
A preprint from arXiv proposes a theoretical framework for embedding cognitive architecture natively into LLMs rather than simulating it via prompt engineering and context management. The framework introduces three mechanisms: Structural Tension (an endogenous loss function from information-manifold conflict), an Offline Recurrent Loop (sandboxed self-processing without external input), and Inference-time Plasticity (topology reconfiguration without weight modification). The authors argue these mechanisms could produce heterogeneous model instances with distinct topological structures through path-dependent evolution, while remaining within governance rails. The paper is primarily theoretical, offering operational definitions, reconfiguration operators, and falsification criteria rather than empirical results.
Semantic persistence model for LLM workflows as inspectable knowledge objects
A new arXiv preprint proposes a conceptual model for treating LLM workflow components — definitions, instances, inference records, context snapshots — as persistent, inspectable knowledge objects in a shared substrate. The framework draws on Lisp-inspired ideas (symbolic forms, object identity, live-image thinking) as explanatory lenses rather than implementation commitments. A central distinction is drawn between 'derive' (deterministic computation) and 'infer' (LLM-mediated judgment under declared context), with formal transition semantics left as future work. The work addresses a gap in how agentic workflow systems represent and reason about their own execution state.
LLMs Show Inverted Compositional Strengths vs. Humans on Reference Resolution Task
This paper evaluates LLMs and humans on the Personal Relation Task (Paperno 2022), distinguishing between Extensional tasks (resolving what an expression refers to) and Intensional tasks (representing structured sense/formula). The study finds that humans outperform LLMs on Extensional tasks while LLMs outperform humans on Intensional tasks—an inverted pattern of strengths. The authors argue this asymmetry reflects the absence of referential grounding in LLM training as a key gap in human-like language understanding.
Survey proposes four-layer architecture for token-operations-oriented LLM inference optimization
A new arXiv preprint introduces a four-layer technical architecture—Multi-model Fusion, Model Optimization, Compute-Model Fusion, and Compute-Network-Model Fusion—for systematically organizing LLM inference optimization techniques. The paper reviews key technologies and industry status at each layer and analyzes their application in real-world business scenarios. The framing around 'token operations' positions inference optimization as an operational discipline analogous to traditional IT operations.