Entity · paper

Form, Not Content? A Preregistered, Placebo-Controlled Evaluation of Learned Error-Conditioned Self-Repair Through Prompts and Weights in Frozen Small Code Models

paperactiveprovisional

form-not-content-a-preregistered-placebo-controlled-evaluation-of-learned-error-conditioned-self-repair-through-prompts-and-weights-in-frozen-small-code-models-c6330121

·1 events·first seen 6d ago

Aliases: Form, Not Content? A Preregistered, Placebo-Controlled Evaluation of Learned Error-Conditioned Self-Repair Through Prompts and Weights in Frozen Small Code Models

Co-occurring entities

PoPE (Popperian Placebo-controlled Evaluation)

More like this (12)

Selection Without Signal, Recovery Through Expression: A Measurement Study of Post-Hoc Falsification Operators for Frozen Small Code Models Rubric-Conditioned Self-Distillation Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QA Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision Self-Preference Is Weak or Absent in Verifiable Instruction-Following Revision: A Four-Model Test Under Genuine Authorship Scaffold, Not Vocabulary? A Controlled, Two-Tier, Pre-Registered Study of a Popperian Code-Generation Skill On the Limits of Prompt-Conditioned Language Models as General-Purpose Learners Error-Conditioned Neural Solvers When Built-in Thinking Helps and Hurts: Constraint-Level Error Shifts in Instruction Following embodiment-aware prompt conditioning PoPE (Popperian Placebo-controlled Evaluation)RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models

Recent events (1)

5arXiv · cs.LG·6d ago·source ↗

PoPE: Placebo-controlled evaluation of error-conditioned self-repair in frozen small code LLMs

A preregistered arXiv preprint introduces PoPE (Popperian Placebo-controlled Evaluation), a methodology for rigorously measuring whether execution error feedback actually improves code generation in frozen small LLMs (0.5–1.5B parameters). The study uses channel-specific placebos—ablating error content while preserving scaffold form—to test both prompt-based and adapter-based self-repair. Results across both channels failed to confirm content-attributable superiority over placebos or baselines, suggesting that apparent self-repair gains in prior work may reflect form rather than semantic error content. The paper argues that writing oracle-derived representations back into generation state replaces testing with conditioning.

Evaluation and Benchmarking Agent and Tool Ecosystem PoPE (Popperian Placebo-controlled Evaluation)Form, Not Content? A Preregistered, Placebo-Controlled Evaluation of Learned Error-Conditioned Self-Repair Through Prompts and Weights in Frozen Small Code Models