Anthropic staff say Claude makes them 12x faster. METR measured the lab moving about 2x faster. Here's what the gap teaches you

Published November 7, 20265 min read

METR, the nonprofit that evaluates frontier models, published its independent assessment of AI R&D acceleration at Anthropic on November 5. It ran from August 12 to October 29 with about 2.5 full-time METR staff. Anthropic gave METR datasets, documents, and interviews, but no direct system access and no query access to unreleased models.

We read the report and METR's earlier productivity studies. We have no access to Anthropic's data, and much of the strongest evidence in the report is redacted, which METR says up front.

What METR found

METR asked three questions. Has Anthropic's rate of capability growth sped up? How much of that is due to AI? And is anything likely to slow it down soon? Its answers, which it calls subjective:

• About 2x. As of October 2026, Anthropic is “likely improving the capabilities of their models on well-specified tasks at around twice the rate” it managed from March 2024 to October 2025. METR wouldn't be surprised by anything from 1.3x to 3x.
• Mostly AI. METR estimates about 80% of the current acceleration comes from Anthropic's use of AI in its own research. It thinks non-AI factors, hiring among them, drove much of the jump in late 2025.
• No clear brake. METR found no convincing bottlenecks or steep diminishing returns. It holds that view with low confidence, because its picture of bottlenecks changed several times during the work.

The core number comes from AECI, Anthropic's version of the Epoch Capabilities Index, which combines many benchmarks into one score. Before October 2025 it rose about 15 points a year. Since then it has risen about 29 points a year, a ratio of 1.9x with a 95% interval of 1.5x to 2.8x.

Three numbers that measure different things

The report contains a capability trend, staff polls, and interviews. They answer different questions, and mixing them up is how most AI productivity claims go wrong.

METR is blunt about the polls. It expects “the responses come from an unrepresentative subset of staff,” notes that self-reports “can overestimate” real impact, and says “individual uplift/speedup estimates do not correspond to acceleration rates.” A researcher who finishes coding tasks 12 times faster still waits on training runs, review, and decisions about what to try next.

The interviews show the same spread. One researcher put AI's share of the slope change at about 15 to 20%, another at 50 to 80%.

What METR's coding studies found

The report can't say whether the 12x polls are accurate, because nothing in it measures individual speedup directly. METR's earlier coding studies did, and in them self-reported and measured speedups often diverged:

The follow-up fell apart for an instructive reason. Many developers refused to join because they “do not wish to work without AI,” and 30 to 50% said they skipped submitting tasks they didn't want to do by hand. Developers running several agents at once also struggled to log their time.

The transcript analysis names the other traps. Task substitution means doing extra, lower-value work because it's cheap now. Task selection means using AI only where it shines. Both inflate a per-task speedup without changing how much valuable work ships.

How to measure your own team

The report keeps self-reports and measured outcomes apart, and that's the habit worth copying. With AECI, METR picked an output measure, fit the trend before the change, and looked for a change in slope.

1. Choose an output, not a feeling. Lead time from first commit to deploy, merged pull requests that don't get reverted within 30 days, or incidents per release. Our guide to Copilot's agent metrics covers why merge rate alone misleads.
2. Use at least a year of history. METR's baseline covers 19 months before its breakpoint. A single sprint before and after a tool rollout is mostly noise.
3. Split well-specified from open-ended work. METR's acceleration is on “well-specified tasks that are easy to grade automatically.” Bug fixes with a failing test and migrations behave differently from design work, so track them separately.
4. Count the work that wouldn't have happened. If agents now write internal tools nobody asked for, that time looks like speed and isn't output.
5. Ask anyway, but label it. Self-reported speedup is still useful for spotting which tasks people hand to agents. Just don't report it as productivity.

What to discount

Anthropic had unilateral redaction rights and used them on most of the evidence METR found most informative, including intermediate capability measures and bottleneck details. Anthropic also cancelled, partway through, a planned check of how it splits compute between capabilities and safety work. METR says readers have to rely heavily on its judgment as a result.

METR discloses that Claude was heavily involved in its own data collection, analysis, and writing. One team member held equity in a competitor and had discussed a role at Anthropic, so their work was checked by a colleague. Anthropic supplied free tokens and laptops, and METR took no payment.

The estimate is also sensitive to modelling choices. Swapping the statistical link function moves the AECI ratio anywhere from 1.1x to 2.9x, though METR's main choice and most alternatives land near 2x.

What it means if you build on Claude

If METR is right, the company making the model you code with is improving it faster than it did a year ago, with no clear slowdown in sight. Expect shorter gaps between releases, and keep your evals, prompts, and model pins in a state where you can test a new version in an afternoon. Our look at AI coding productivity data covers the wider research on what those models do for working teams.

Sources

• METR: Independent Assessment of AI R&D Acceleration at Anthropic in October 2026 (November 5, 2026): acceleration estimates, AECI figures, polls, interviews, redactions, and conflicts of interest.
• METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity: the 19% slowdown trial.
• METR: We are Changing our Developer Productivity Experiment Design (February 24, 2026): late-2025 estimates and selection problems.
• METR: Analyzing coding agent transcripts to upper bound productivity gains (February 17, 2026): the Claude Code transcript study.