Anthropic AI Progress Nearly Doubles: METR’s Findings

Anthropic’s AI progress comes with a disclosure gap: some of the evidence explaining its speed is confidential.

In a November 5 assessment, Model Evaluation and Threat Research (METR) estimated that Anthropic’s benchmark scores have improved at 1.9 times their earlier rate since October 2025. The broader finding concerns capabilities on well-specified tasks, with substantial uncertainty about how far those gains extend.

For IT buyers, the distinction matters: faster benchmark progress does not establish how much a new model will improve their own workflows.

The numbers behind the speedup

METR’s 11-week evaluation examined internal research data, including benchmark results for unreleased models, and roughly 15 staff interviews. AECI is Anthropic’s version of the Epoch Capabilities Index. METR judged AI use likely responsible for most current acceleration, while acknowledging other contributing factors.

“We are confident that the acceleration was not due to an acceleration in the scaling of training compute,” METR said in its report.

Informal Slack polls put average self-reported speedup from the most capable internal model at approximately 12x in October. METR considered this weak evidence from a likely unrepresentative sample, rather than a measured company-wide productivity gain.

The transparency problem

Anthropic exercised its redaction rights to remove what METR called “a majority of the evidence we found most informative for these conclusions.” The company described the redactions as “aimed at removing confidential information that could accelerate other AI developers.”

The findings also provide context for Anthropic CEO Dario Amodei’s call to slow frontier AI development. His essay argued that additional time could help developers strengthen safeguards. The assessment does not establish whether Anthropic met or missed a specific pacing commitment.

More must-read AI coverage

What IT buyers should check before upgrading

For organizations evaluating AI tools, the useful question is whether a newer model completes their work more reliably and economically. A vendor’s development speed should inform what buyers investigate, rather than determine purchasing decisions.

Build a repeatable evaluation around representative tasks. Include straightforward requests and difficult cases involving incomplete information, conflicting instructions, or actions that require human approval.

For development teams, research on AI coding agents offers useful context for designing evaluations. Track review effort, defects, and accepted changes alongside output volume; producing more code is only one part of the result.

Apply the same scrutiny to independent assessments, including proposals for rival AI companies to test one another’s models. Ask vendors which model versions were evaluated, what access evaluators received, which conclusions remain uncertain, and what supporting evidence buyers can inspect.

Keep evaluation results tied to specific model versions and configurations. Before expanding deployment, set acceptance criteria, assign responsibility for reviewing failures, and establish a rollback option if quality deteriorates.

The purchasing decision should rest on demonstrated improvements in your environment. Faster development makes a repeatable testing process more useful, especially when public evidence cannot answer every question.

Read more: Put model evaluations in context with this guide to Claude’s models, tools, pricing, and privacy controls.