FrontierPhysics: Claude Opus 5.5 Clears Just 27% of Tasks

The best-performing AI agent configuration passed just 27.4% of tasks on FrontierPhysics, a new benchmark that evaluates frontier models against end-to-end physics research problems built by working physicists. Claude Opus 5.5 leads across all 26 tested configurations, but no setup cleared more than 46 of 168 rollouts. Manual failure analysis reveals that gaps in physics knowledge account for only 11% of errors, while non-rigorous reasoning and missing expert intuition drive most failures.

What FrontierPhysics Measures

FrontierPhysics ships 56 tasks built by 42 physicists from their own published research across nine subfields, including condensed matter physics, atomic and molecular optics, astrophysics, fluid dynamics, and high energy physics. Each task requires agents to review existing literature, develop a research plan, conduct experiments or simulations, and produce both structured outputs and a scientific manuscript. The benchmark uses a deterministic verifier to check numerical results and an expert-authored rubric with binary blocker gates that zero the entire score if scientific validity fails.

To validate their grading system, the authors scored their own published papers against the rubric, achieving 96.2% on average. Eighteen of the 56 tasks are drawn from published work, four of which appeared in Nature.

Claude Opus 5.5 Leads at 27.4%

Across 22 frontier models tested in 26 different configurations, Claude Opus 5.5 delivered the highest performance at 27.4%, successfully completing 46 of 168 rollouts. No other configuration exceeded this result. The evaluation spanned multiple model families and deployment approaches, but the gap between the best performer and perfect completion remains wide.

The benchmark's structure treats each task as a complete research workflow, not a multiple-choice quiz or code generation exercise. Agents must demonstrate scientific reasoning, experimental design competence, and the ability to validate their own work against expert standards.

Failure Analysis: Reasoning Over Knowledge

The research team conducted manual trajectory analysis of 168 GPT-6 Astra rollouts to understand failure patterns. The results challenge assumptions about what limits frontier agent performance on scientific tasks:

• 48% of failures stemmed from non-rigorous reasoning, including circular self-verification and failure to consider edge cases
• 34% traced to missing physics expert intuition, such as inability to identify error sources and poor debugging decisions
• 11% involved physics knowledge gaps
• 5% were execution errors

The paper states plainly that frontier agents' primary weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.

Release Date and Availability

FrontierPhysics was officially released on November 5, 2026. The benchmark and evaluation protocol are now available to researchers evaluating frontier model capabilities on scientific reasoning tasks.

What This Means

FrontierPhysics adds a data point to ongoing debates about frontier model scientific capabilities. The 27.4% success rate from the best configuration suggests current agents struggle most with the meta-skills of scientific work: rigorous reasoning, intuition about where errors hide, and knowing which edge cases matter. These findings arrive during a high-activity period for agent evaluation, with over 530 agent-related stories logged in the last 90 days. The benchmark offers researchers a concrete way to measure progress on scientific reasoning rather than knowledge retrieval or code generation alone.