METR Says Anthropic's AI Progress Is Running Twice as Fast — With Most Evidence Redacted

Anthropic's AI models have been improving about 1.9 times faster than their old pace since October 2025, according to an independent assessment by METR published on 5 November 2026. The catch is in the fine print: Anthropic redacted most of the evidence METR found most informative, so the public has to take the headline number largely on METR's word. The finding and the uncertainty should never be reported apart.

METR is a nonprofit research group that evaluates AI systems. Anthropic gave it unusual access for this assessment: eleven weeks inside the company, from 12 August to 29 October 2026, with internal data on pretraining, reinforcement learning, training compute, benchmark scores for internal models, and interviews with staff. METR says no payment changed hands. The team was small — 2.5 full-time equivalents — and conducted about 15 interviews.

The headline measure is AECI, Anthropic's version of the Epoch Capabilities Index — a benchmark-based index of what its models can do on well-specified tasks. METR's estimate is that AECI scores have been growing at 1.9 times the pre-October 2025 rate, with a 95% confidence interval of 1.5 to 2.8 times. METR says Anthropic made roughly two years of historical progress in one year — as much progress on benchmarks in the past year as it made between October 2023 and October 2025, roughly the span from Claude 2.1 to Claude Sonnet 4.5.

Researchers working on laptops in a lab with a rising chart on a wall screen
Illustration created for  INORYVA


The 1.9x number, and how wide the uncertainty is

A single number hides a wide range. METR's own judgement band is broader than the statistical interval: it would not be surprised if the true acceleration were closer to 1.3 times or 3 times the historical rate. The 1.9x figure is the central estimate, not a precise measurement.

It is also higher than the figure Anthropic itself published. The Claude Haiku 5.5 system card reported 1.7 times, and METR says its estimate is greater mainly because it includes data on newer internal models that Anthropic's public figure did not. That detail matters: the acceleration METR measured is partly driven by models the public has never seen.

FigureWhat METR reported
Capability pace since Oct 2025~1.9x the pre-Oct 2025 rate
95% confidence interval1.5x to 2.8x
METR judgement band1.3x to 3x
Anthropic's own figure (Haiku 5.5 card)1.7x
Share of acceleration attributed to AI R&D~80% (subjective, high uncertainty)
Assessment window12 Aug – 29 Oct 2026 (11 weeks)

For context, this is a research assessment, not a product announcement. It sits in the same research family as the State of AI Report 2026, the broadest industry overview on this blog, and it should be read with the same caution as any single-lab claim — including the earlier AI breakthrough complexity theory claim from OpenAI and Anthropic, which was also reported with its verification limits stated.

What is driving the speedup, in METR's judgement

METR's subjective estimate is that a majority of the acceleration — perhaps about 80% — is due to AI doing research and development work. That number comes with substantial uncertainty attached, and METR is explicit about it. It is a judgement, not a measurement, and it should always be quoted with the uncertainty attached.

Two things METR is more confident about. First, the acceleration is happening mainly through AI assistance rather than fully autonomous contributions: researchers use AI to speed up their work, but AI is not yet making highly significant, fully autonomous contributions to the most important parts of model training. METR notes "signs of life" that autonomous contributions could grow soon, but that is a forecast, not a finding. Second, the acceleration was not caused by faster growth in training compute. METR says it was able to confidently determine that training-compute scaling did not accelerate — the speedup came from elsewhere in the process.

Researcher pointing at abstract charts and arrows on a whiteboard
Illustration created for INORYVA


The redaction problem

The most important sentence in METR's assessment may be the one about what readers cannot see. METR writes that Anthropic "redacted a majority of the evidence we found most informative for these conclusions" and that Anthropic described the redactions as aimed at removing confidential information that could accelerate other AI developers. METR's consequence is blunt: "the public has to rely on METR's judgment much more than would be ideal."

That is the honest frame for the whole story. The assessment is independent — METR was not paid, and it publishes its methods and uncertainty — but the underlying evidence is largely private. A reader who quotes 1.9x without the redaction caveat overstates what can be checked; a reader who dismisses the finding entirely ignores that an independent evaluator with deep access reached the same direction as Anthropic's own public figure, only higher.

The unreleased model behind the number

Part of the reason METR's figure exceeds Anthropic's public one is an internal-only model METR calls Model 4, which finished training in September–October 2026. METR says Model 4 scored 178.6 on AECI, rising to 179.7 when three suspicious benchmarks are excluded, and scored above 50% on every AECI benchmark on which it was tested. Model 4 is not a product: METR describes it as internal-only, and Anthropic has not announced a model by that name. It should never be presented as a forthcoming release, and no release date, price, or capability claim beyond the AECI score should be attached to it.

The Model 4 detail is also a reminder of how much of this assessment rests on things the public cannot verify. The strongest evidence for the acceleration comes from models and data that remain inside Anthropic, so no outside lab can re-run METR's numbers without the same access.

A separate question: the "2x Acceleration Threshold"

Do not confuse the 1.9x pace with a different METR document published the same day. In a separate review of the "Risks from automated R&D" section of Anthropic's August 2026 Risk Report, METR examined whether Anthropic's models as of 15 July 2026 had crossed the company's early-warning "2x Acceleration Threshold" — a test against a no-AI counterfactual and the fastest sustained past rate, not the same test as the 1.9x pace figure.

METR's verdict there is carefully worded: it agrees with Anthropic that catastrophic risks from the July 15th models were low, but it thinks it is plausible that the models had crossed the early-warning threshold at the time. Most relevant researchers at METR and Anthropic lean toward the threshold not having been crossed, but METR says the claim that it was not crossed "relies on contestable judgment calls that aren't communicated clearly in the public report." In METR's view, the public report understates the level of acceleration that was occurring. "Plausible" is the operative word — this is not a finding that the threshold was crossed.

The distinction matters for readers following Anthropic's safety reporting. The October assessment measures how fast capabilities are improving. The threshold review judges whether a specific safety tripwire was crossed in July. They are different questions, answered in different documents, and merging them would misstate both.

Two researchers discussing a line graph on a laptop
Illustration created for INORYVA


What this does not prove

The assessment does not prove that AI is now doing research on its own. METR's own words are that R&D is accelerated "primarily via AI assistance rather than fully autonomous contributions." It does not prove that the acceleration will continue at the same rate, or that it applies to other labs — this is an assessment of Anthropic alone, over one eleven-week window, against its own history. And it does not prove that faster capability growth means faster progress toward any particular milestone; AECI measures performance on well-specified benchmark tasks, not general intelligence.

For research context, see Anthropic's Claude AI drug discovery programme and the separate questions in the Claude rogue behaviour report.

Key Takeaways

  • METR estimates Anthropic's AI capability pace at ~1.9x since October 2025 (95% CI 1.5–2.8x; judgement band 1.3–3x) — roughly two years of historical progress in one year.
  • METR's ~80% attribution of the acceleration to AI doing R&D is a subjective estimate with substantial uncertainty; the work is assistive, not fully autonomous, and not driven by faster training-compute growth.
  • Anthropic redacted most of the most informative evidence, so the public must rely heavily on METR's judgement — quote the number with the caveat, never alone.
  • Model 4 (AECI 178.6) is an internal-only model, not a product; never present it as a release.
  • The "2x Acceleration Threshold" question is separate: METR says crossing it in July 2026 is plausible, not proven, and most researchers lean toward it not having been crossed.

Conclusion

The METR assessment is the most detailed independent look yet at how fast one leading lab is improving — and the most honest about its own limits. The 1.9x figure is worth reporting because an unpaid evaluator with deep access reached it, and worth qualifying because the evidence behind it is mostly redacted. The right takeaway is not that AI progress has doubled, but that the best available measurement says it roughly doubled at Anthropic, with wide error bars and much of the proof kept private.

Sources

Facts checked 6 November 2026 against METR's own pages. "Model 4 finished training Sept–Oct 2026" (not October only); METR calls it "internal-only", not "unreleased" in its own wording.

Comments

Post a Comment