specfirst-3be0e70d·1 events·first seen Aliases: SpecFirst
SpecFirst is a two-stage LLM agent framework that treats behavioral specification elicitation as a distinct first-class phase before code synthesis, inspired by classical requirements engineering. A dedicated spec agent probes an execute-only binary oracle and combines observations with documentation into a structured specification, which a separate synthesis agent then uses for implementation. Evaluated on all 200 instances of ProgramBench across four models, SpecFirst improves test pass rates by 6.9–21.3% and binary exploration coverage by 9.4–18.5% over single-loop baselines. The work addresses a known gap where frontier models solve fewer than 1% of from-scratch programming tasks on ProgramBench.