Almanac
← Events
7OpenAI Blog·1mo ago

OpenAI Introduces FrontierScience Benchmark for Scientific Research Tasks

OpenAI has released FrontierScience, a new benchmark designed to evaluate AI reasoning capabilities across physics, chemistry, and biology. The benchmark is intended to measure progress toward AI systems capable of performing real scientific research tasks. This represents OpenAI's effort to establish a rigorous evaluation framework for frontier-level scientific reasoning, going beyond standard academic problem sets.

Related guides (3)

Related events (8)

6Openai Blog·3d ago·source ↗

OpenAI introduces LifeSciBench, a life sciences AI evaluation benchmark

OpenAI has released LifeSciBench, a benchmark designed to evaluate AI systems on real-world life science research tasks and decisions. The benchmark is described as expert-authored and expert-reviewed, targeting domain-specific evaluation in biology and related fields. This addresses a gap in specialized scientific benchmarking for AI systems.

7Openai Blog·1mo ago·source ↗

PaperBench: OpenAI Benchmark for Evaluating AI Agents on Research Replication

OpenAI introduces PaperBench, a benchmark designed to evaluate AI agents' ability to replicate state-of-the-art AI research papers end-to-end. The benchmark targets a high-complexity capability: reproducing experimental results from frontier AI research, which requires code generation, experimental design, and scientific reasoning. This positions PaperBench as a tool for tracking progress toward autonomous AI research agents.

5Ai Snake Oil·1mo ago·source ↗

Open-world evaluations for measuring frontier AI capabilities: Introducing CRUX

This commentary introduces CRUX, a new evaluation project designed to assess frontier AI systems on long-horizon, open-ended, and messy real-world tasks. The piece argues that existing benchmarks are insufficient for capturing the full range of capabilities exhibited by frontier models in complex settings. CRUX aims to fill this gap by providing evaluations that better reflect deployment-relevant performance.

7Openai Blog·1mo ago·source ↗

Introducing OpenAI Frontier

OpenAI has launched OpenAI Frontier, an enterprise platform designed for building, deploying, and managing AI agents. The platform provides shared context, onboarding workflows, permissions management, and governance tooling. This positions OpenAI more directly in the enterprise AI infrastructure and agent orchestration market.

6arXiv · cs.AI·12d ago·source ↗

AARRI-Bench evaluates frontier LLMs and agents on granular research-intern-level tasks

Researchers introduce AARR (Act As a Real Researcher), a new benchmark series targeting whether AI agents can emulate the professionalism, thoroughness, and nuanced judgment of human researchers in granular research scenarios—not just macro-level task execution. The first benchmark, AARRI-Bench, tests frontier models and agentic harnesses, finding that even the best configuration (Mini-SWE-Agent with Claude Opus 4.7) achieves only 68.3% success, frequently missing subtle but critical details obvious to human researchers. The work argues that closing the gap requires deeper modeling of research behavior rather than more complex scaffolding.

6Openai Blog·1mo ago·source ↗

Introducing HealthBench

OpenAI has released HealthBench, a new evaluation benchmark designed to assess AI model performance and safety in healthcare settings. The benchmark was developed with input from over 250 physicians and targets realistic clinical scenarios. It aims to establish a shared standard for measuring how well AI models handle health-related tasks.

5Openai Blog·1mo ago·source ↗

Frontier AI regulation: Managing emerging risks to public safety

OpenAI published a policy position on regulating frontier AI systems, focusing on managing emerging risks to public safety. The piece outlines OpenAI's perspective on how governments and regulatory bodies should approach oversight of the most capable AI models. This represents a formal public stance from a leading AI lab on the shape of future AI governance frameworks.

5Interconnects·1mo ago·source ↗

Opus 4.6, Codex 5.3, and the post-benchmark era

A Interconnects commentary piece examining how to compare frontier AI models in 2026, using Anthropic's Opus 4.6 and OpenAI's Codex 5.3 as case studies. The piece appears to argue that traditional benchmarks are no longer sufficient for distinguishing model capabilities at the frontier. This reflects a broader industry shift toward more nuanced, task-specific evaluation methods.