5arXiv cs.AI (Artificial Intelligence)·
ProjAgent introduces procedural similarity retrieval for repository-level code generation
ProjAgent is a new repository-level code generation system that adds procedural similarity as an explicit retrieval signal, decomposing target functions into intermediate reasoning steps to find analogous implementations across a codebase. It combines this with conventional semantic retrieval and a static-analysis feedback loop for iterative code repair. Evaluated on the REPOCOD benchmark, ProjAgent achieves 41.14% Pass@1, outperforming existing retrieval-based baselines. The work identifies procedural similarity as a previously unexplored retrieval dimension for this class of coding agents.
Related guides (2)
Related events (8)
ReproRepo: Scalable LLM agent framework for reproducibility auditing using GitHub issues
ReproRepo is a new framework for evaluating LLM agents on reproducibility auditing of ML research, using naturally occurring GitHub issues as supervision signals rather than costly manual curation. The framework is instantiated on 1,149 recent ML papers from major conferences and benchmarks four frontier model-agent configurations. The best-performing agent (Codex with GPT-5.5) surfaces at least one semantically related human-reported reproduction blocker for ~90% of papers, though exact localization of issues remains a weakness. The work provides a reusable, scalable evaluation harness for this underexplored agentic task.
Probe-and-Refine Tuning improves coding agent performance via iterative repository guidance refinement
A new arXiv paper introduces probe-and-refine tuning, a procedure that uses synthetic bug-fix probes to iteratively improve AGENTS.md repository guidance files for LLM-based coding agents without requiring an agent loop during tuning. Evaluated on SWE-bench Verified with Qwen3.5-35B-A3B, the method achieves 33.0% mean resolve rate versus 28.3% for a static knowledge base baseline and 25.5% for an unguided baseline. The improvement is attributed to coverage gains—refined guidance helps agents locate the correct files rather than improving patch quality—and a step-budget experiment shows guidance is necessary for agents to productively use larger compute budgets.
SWE-Explore: New benchmark isolates repository exploration capability in coding agents
SWE-Explore is a new benchmark targeting repository exploration as a distinct, fine-grained capability of coding agents, separate from end-to-end task resolution. It covers 848 issues across 10 programming languages and 203 open-source repositories, with line-level ground truth derived from successful agent trajectories. Evaluation across retrieval methods, coding agents, and specialized localizers finds that agentic explorers outperform classical retrieval, and that line-level coverage and efficient ranking remain the key differentiators at the frontier. The benchmark addresses a gap in SWE-bench-style evaluations that treat task resolution as a binary outcome.
Recursive Agent Harnesses (RAH): harness recursion extends model recursion for long-context coding agents
A new arXiv preprint introduces the Recursive Agent Harness (RAH), a pattern where a parent agent generates executable scripts that spawn parallel subagent harnesses with filesystem tools, code execution, and planning capabilities. The authors frame this as 'harness recursion', a code-first extension of model recursion from recursive language models. Evaluated on the Oolong-Synthetic long-context benchmark, RAH improves over the Codex coding-agent baseline from 71.75% to 81.36% with GPT-5 as backbone, and reaches 89.77% with Claude Sonnet 4.5. The work connects emerging production patterns (e.g., Anthropic's dynamic workflows) to a formal architectural concept.
Audit finds GSO, SWE-Perf, and SWE-fficiency benchmarks unreliable for measuring coding agent progress
A new arXiv paper audits three prominent repository-level code-optimization benchmarks (GSO, SWE-Perf, SWE-fficiency) used to rank coding agents, finding significant reliability problems across all three. Reference patches satisfy validity rules in cross-machine replays for only 39/102 GSO tasks and 11/140 SWE-Perf tasks, and leaderboard rankings disagree on 9 of 28 pairwise comparisons depending on scoring rule choice. The authors also find that at least one public submission already matches or beats the reference patch on 85.3% of replay-valid tasks, suggesting aggregate leaderboard scores obscure the true frontier. The study raises substantive concerns about whether these benchmarks are providing reliable signal for claims of coding-agent capability progress.
Large-scale study finds autonomous coding agents concentrate repository-level integration risk at twice the rate of human contributors
A new arXiv paper analyzes over 930,000 agent-authored pull requests to measure 'integration friction' — the cost of merging contributions into concurrently-changing codebases. The study finds that roughly half of friction variation is a persistent property of the repository rather than any individual contribution or agent, and that agent-authored contributions concentrate this repository-level friction at approximately twice the rate of human contributions (intraclass correlation 0.30 vs. 0.16). The authors argue this means AI-native software risk is an ecosystem-level phenomenon and should be governed and evaluated at the repository level rather than agent-by-agent.
RePro: Retrospective Progress-Aware Self-Refinement for LLM Agent Training
Researchers introduce RePro (Retrospective Progress-Aware Training), a framework addressing the gap between step-wise RL optimization and metacognitive task-progress awareness in LLM agents. The approach uses a forward-then-reflect rollout paradigm where agents execute actions online and then retrospectively assess step-wise progress given the completed trajectory and known outcome. Evaluated on WebShop, ALFWorld, and Sokoban, RePro achieves up to 12% absolute success rate gains over baseline Qwen-family models without requiring continuous external supervision.
Reversa: A Multi-Agent Framework for Reverse Engineering Legacy Software into AI-Readable Operational Specifications
Reversa is a multi-agent pipeline framework that converts legacy software systems into traceable operational specifications suitable for use by AI coding agents. The framework employs specialized agents for surface mapping, module analysis, implicit rule extraction, architecture synthesis, and specification review, with mechanisms for traceability, confidence marking, and gap preservation. An exploratory case study on migrating an ATM system from COBOL to Go produced 517 confidence-indexed claims, 53 Gherkin parity scenarios, and a partial reconstruction plan, though final validation was not completed. The system is distributed as a Node.js CLI and is positioned relative to literature on reverse engineering, LLM-based documentation, and software agents.