What Terminal-Bench is
Terminal-Bench is a benchmark designed to evaluate AI agents on long-horizon, multi-step tasks executed inside a real terminal environment. Where earlier coding benchmarks measured whether a model could complete a function or patch a specific bug, Terminal-Bench asks whether an agent can autonomously navigate a shell, chain tool calls, manage state across many steps, and recover from failures — the kind of work a software engineer actually does over hours, not seconds.
The benchmark has been versioned as frontier capability has advanced: the event bundle references Terminal-Bench (original), Terminal-Bench 2.0, Terminal-Bench V2, and Terminal-Bench 2.1, with each iteration raising the ceiling of what constitutes a meaningful score.
Why it matters
Terminal-Bench has emerged as one of the primary leaderboards distinguishing frontier agentic coding models from each other. Because it demands sustained autonomous operation rather than single-shot generation, it is more sensitive to the failure modes that matter most in production: context loss over long trajectories, inability to recover from unexpected states, and degraded planning as task complexity compounds. Labs including Anthropic, OpenAI, and DeepSeek all cite Terminal-Bench scores in major model releases, and it appears alongside SWE-Bench Verified as a standard dual-axis evaluation for software engineering capability.
Score progression and the frontier race
The score trajectory across the event bundle tells a clear story of rapid escalation:
- Claude Opus 4 (September 2025) scored 43.2% on Terminal-Bench at launch, establishing it as an early frontier reference.
- Claude Opus 4.6 (March 2026) claimed the top score on Terminal-Bench 2.0 at its release, though the exact percentage was not disclosed in the events.
- Claude Mythos Preview (April 2026) scored 82% on Terminal-Bench 2.0 — a restricted-access model that Anthropic assembled a $100M vulnerability-patching consortium around before broader deployment.
- GPT-5.6 Sol (July 2026) achieved 91.9% on Terminal-Bench 2.1 in a government-restricted preview, the highest score reported in the current bundle, while approaching Claude Mythos 5 on the related ExploitBench.
DeepSeek V3.1 also reported benchmark gains on Terminal-Bench at its release, though exact scores were not disclosed.
Research use: probing agent failure modes
Beyond leaderboard competition, Terminal-Bench has become a standard testbed for research into the mechanisms of agentic failure and improvement:
Memory and context decay. The Proactive Memory Agent paper evaluated on Terminal-Bench 2.0, finding that a plug-and-play memory module yielding selective reminder injection produced a +8.3 percentage-point gain in pass@1 — confirming that "behavioral state decay" (task-critical context getting buried in long trajectories) is a primary failure mode the benchmark exposes.
Verification as a scaling axis. The LLM-as-a-Verifier framework achieved 86.5% on Terminal-Bench V2 without any additional model training, using continuous scores derived from token logit distributions rather than discrete judge outputs. This result suggests that verification quality — not just model capability — is a meaningful lever for Terminal-Bench performance.
Open-weight training recipes. The Tmax paper demonstrated that a 9B parameter model trained with an open RL recipe (combining difficulty-controlled data generation, persona diversification, and outcome-only RL) could reach 27% on Terminal-Bench 2.0, outperforming larger models from prior work. This lowers the barrier for academic research on terminal agents and establishes Terminal-Bench as a tractable target for smaller-scale compute.
Benchmark integrity concerns
The Auto Benchmark Audit (ABA) framework, applied to 168 benchmarks across nine domains, found critical issues — ambiguous specifications, environment conflicts, incorrect ground truths — in over 25.7% of evaluated tasks. Filtering out flawed tasks improved average performance on Terminal-Bench 2 by 9.6%, indicating that current raw leaderboard scores are meaningfully distorted by task quality problems. This is a known challenge for any benchmark that involves real environment execution, where ground-truth verification is harder than in closed-form tasks.
Relationship to adjacent benchmarks
Terminal-Bench consistently appears alongside SWE-Bench Verified as a paired evaluation: SWE-Bench probes issue resolution in existing codebases, while Terminal-Bench probes sustained autonomous terminal operation. Models that lead on one tend to lead on the other, but the correlation is imperfect — the esoteric programming language study found that capability stratification visible on Terminal-Bench is compressed into narrow bands on SWE-Bench Verified, suggesting the two benchmarks capture partially distinct skills.
Other benchmarks cited alongside Terminal-Bench in frontier model releases include Humanity's Last Exam, GDPval-AA, BrowseComp, CyberGym, and GPQA Diamond — Terminal-Bench's role in this suite is specifically to anchor the agentic software engineering axis.
Where it's heading
The versioning cadence (1.0 → 2.0 → 2.1) and the rapid score compression at the top — from 43% to 91% in under two years — suggest Terminal-Bench will need continued evolution to remain discriminative at the frontier. The research literature is already pointing toward the next hard problems: effective recall over very long trajectories, recovery from compacted context, and whether external memory architectures can substitute for raw context length. These are the axes on which future Terminal-Bench versions are likely to differentiate top models.
