5arXiv cs.LG (Machine Learning)·
Paper proposes 'vocabulary gap' and 'verifier gap' framework for open-ended AI innovation
A new arXiv preprint argues that current AI systems are fundamentally limited by fixed representational frames — they search within a given conceptual vocabulary rather than inventing new primitives. The authors characterize this limitation through two gaps: the vocabulary gap (inability to create and stabilize new representational primitives) and the verifier gap (inability to evaluate a primitive whose value is only apparent after future reuse). They propose a 'ladder of innovation autonomy' and outline architectural directions including persistent memory for invented primitives and adaptive verification mechanisms.
Related guides (2)
Related events (8)
Benchmark gap paper: EU AI Act requires doctrinal legal reasoning evals that don't yet exist
A new arXiv preprint identifies a critical measurement gap in legal AI evaluation: existing benchmarks test paralegal and ancillary tasks rather than doctrinal legal reasoning, which is the interpretive core of legal work. The authors argue this gap is not merely methodological but legally significant, because the EU AI Act's 'appropriate accuracy' requirement for high-risk AI in the judicial domain cannot be operationalized without a doctrinal-reasoning benchmark. The paper proposes a benchmark framework aimed at filling this gap under EU AI Act compliance requirements.
Emergent language in multi-agent RL proposed as generative methodology for studying AI consciousness
A new arXiv preprint proposes using emergent language (EL) in multi-agent reinforcement learning as a generative methodology for studying consciousness-relevant structure in AI systems, contrasting with existing discriminative or architectural approaches. Agents begin with minimal language exposure and develop communication under task pressure alone, aiming to avoid artifacts from human language priors. As a proof of concept, the authors show agents develop self-referential communication including an echo-mismatch detection circuit that emerges from environmental affordances rather than task structure or architecture.
Survey of recursive self-improvement in AI: taxonomy, evaluator hierarchy, and governance gaps
A new arXiv survey covers 1,250 papers (2024–2026) on AI self-improvement, proposing a two-axis taxonomy distinguishing what is improved (behavior, policy, evaluator, or research process) from the degree of loop closure (human-in-the-loop to fully closed). The authors construct a verification hierarchy for self-evaluation signals—from formal verifiers (strongest) to intrinsic self-assessment (weakest)—and find that demonstrated self-improvement strength tracks this hierarchy while failure modes (self-confirming loops, model collapse, diversity collapse) arise from its violations. The paper argues that 'research direction-setting' remains the key bottleneck keeping humans in the loop, and identifies governance-grade measurement of self-improvement as the most underpopulated niche in the field. The work connects technical RSI limits to safety and governance concerns raised by frontier labs experimenting with closed-loop AI research.
AI-Assisted Systematization for Evaluating GenAI Systems
This paper addresses a foundational gap in GenAI evaluation: the underspecification of broad, contested concepts like 'reasoning,' 'fairness,' or 'creativity.' The authors introduce a structured artifact called a 'concept spec' and a validation worksheet, then build two AI-assisted systematizers—a zero-shot approach and a multi-agent approach—to convert vague evaluation targets into measurable, structured accounts. They apply these tools to hate-based rhetoric and digital empathy, assessing the resulting specs on content validity and information recoverability. The work positions AI assistance as a scalable aid for the cognitively demanding process of evaluation design.
Prover-Verifier Games improve legibility of language model outputs
OpenAI presents research on prover-verifier games as a mechanism to improve the legibility and verifiability of language model outputs. The approach frames output generation as a game between a prover (the model producing solutions) and a verifier (checking correctness), incentivizing clearer, more human-auditable reasoning. The work targets a core alignment challenge: ensuring AI-generated solutions are interpretable and trustworthy to both humans and automated systems.
Arxiv paper proposes 'situation perception' as a necessary primitive for ASI
A preprint argues that current LLMs are fundamentally incomplete as paths to artificial superintelligence because they lack 'situation perception': the ability to construct, revise, and act within internal simulations of possible worlds across latent time. The authors identify three required components — abstract prediction, long-term compressed memory, and active learning guided by objectives — and propose tests for measuring progress. The paper frames this as a missing cognitive primitive analogous to developmental milestones in human infants.
Formal theory shows infinite trivial output is provably necessary for AI systems generating valuable mathematics
A new arXiv paper models AI-assisted formal mathematics generation as a nested language-generation-in-the-limit problem, using a proof checker as a membership oracle and an adversarial enumeration of the mathematical literature as the signal for 'valuable' content. The authors prove a sharp dichotomy: generators emitting only finitely many trivial (correct but worthless) statements achieve at most α/2 coverage of unseen valuable mathematics, while allowing an infinite (but asymptotically vanishing) stream of trivia raises the optimum to 1−α/2. The central result is that a perfect verifier cannot substitute for mathematical taste, and the flood of certified-but-trivial output from AI proof systems is a provable mathematical necessity, not an engineering failure. The work formalizes the gap between formal verifiability and mathematical value, which is increasingly the binding constraint as AI-proof-assistant systems scale.
OpAI-Bench: Benchmark for detecting AI text across progressive human-AI co-editing workflows
Researchers introduce OpAI-Bench, a benchmark for studying AI-text detection across progressive human-to-AI document revision workflows, covering document, sentence, token, and span granularities. Starting from human-written documents, the benchmark constructs nine sequentially revised versions per sample under five AI edit operations and varying AI coverage levels across four domains. Key findings include that mixed-authorship intermediate versions are often harder to detect than fully human or heavily AI-edited endpoints, revealing non-monotonic detection patterns absent from existing benchmarks. The work addresses a gap in AI-text detection research as real-world documents increasingly result from iterative human-AI co-editing rather than pure generation.