DAI Anthropic Speeds Up 2x, but METR Says Evidence Is Hidden

[admin]

Anthropic is improving its AI models about twice as fast as it did a year earlier, according to an independent review. The catch: the company redacted most of the evidence the reviewers said mattered most, so the public has to take their word for a finding that could reshape how risky the pace of AI progress looks. The assessment came from Model Evaluation and Threat Research (METR), published on Thursday. METR spent 11 weeks inside Anthropic from mid-August to late October, with access to internal benchmark scores, training metrics, staff interviews and surveys. It took no payment from the lab. The number On Anthropic’s internal capability index (AECI), model scores have been rising at an estimated 1.9 times the pre-October 2025 rate, with a 95% confidence interval of 1.5–2.8x. In plain terms, METR says Anthropic has made roughly two years of historical progress in the past year. Most METR staff on the project now believe the lab is “likely” improving on well-specified tasks at around twice the March 2024–October 2025 pace, though they would not be surprised if the true figure sat closer to 1.3x or 3x. METR says it can “confidently” rule out faster growth in training compute as the cause. Interviews and surveys point instead to heavy use of AI tools to help researchers write code, run experiments and try more ideas. Staff Slack polls of self-reported speedup reached about 12x for the latest internal model in October — loose evidence, METR stresses, but consistent with a large lift. AI is not yet making highly significant, fully autonomous contributions to the hardest training steps, the reviewers were told, though “signs of life” for that are appearing. What you cannot see Anthropic had unilateral rights to redact non-public information from the public report. METR says the company removed a majority of the evidence the team found most informative for its risk conclusions, including detailed acceleration estimates by pipeline stage, bottleneck analysis and forecasts for future models. Anthropic said the cuts were meant to strip confidential information that could help rival developers. METR is blunt about the result: the public cannot judge the evidence itself or reconcile it with other sources. “This report does not achieve ‘transparency about evidence’,” it wrote. Readers must lean on METR’s judgement more than the group thinks is ideal for questions of this weight. AP News noted the same tension: a warning about faster progress, tied to material most people cannot inspect. Why the pace claim is contested The reviewers list ways they could be wrong in either direction. Acceleration might be limited to clean, auto-gradable tasks and overstate messy real-world impact. Anthropic’s earlier AECI growth was slower than the wider industry frontier, which could inflate the “2x” comparison. On the other side, redacted internal metrics hinted that acceleration could be above 2x and still rising, and METR saw no clear bottlenecks that would block further speed-ups over the next year. That matters because recursive self-improvement — AI systems that meaningfully help build the next, faster generation — is the scenario safety researchers worry could compress timelines. METR’s guess is that a majority (perhaps around 80%) of the current acceleration is due to AI assistance, with more uncertainty about late-2025 drivers. Co-founder Sam McCandlish said Anthropic plans its own public analysis of how fast progress is moving inside the company, The Verge reported. Why it matters If frontier labs are already getting a large productivity lift from their own models, outside watchdogs need evidence the public can check — not only headline multiples. A doubling of the progress rate changes how much time societies have to test, regulate and adapt. Redacting the underlying measurements turns an independent assessment into something closer to a trusted intermediary’s opinion, which is useful, but not the same as open verification. For workers already seeing AI reshape day-to-day output — including longer days rather than shorter ones — the question is no longer whether the tools help. It is how fast that help compounds inside the labs that build them, and who gets to see the numbers. The post Anthropic Speeds Up 2x, but METR Says Evidence Is Hidden appeared first on DailyAI.

Source: https://dailyai.com/2026/11/anthropic-s ... is-hidden/