METR time-horizon audit — the 50% horizon is a fitted curve, not a measurement
| Date | 2026-10-11 |
|---|---|
| Category | Benchmarks |
| Judgement | Neutral |
| Gauge weight | 1 / 3 |
| Source | Nguyen and Fithian (Berkeley) statistical audit of the METR time-horizon plot, October 2026, recorded in AGI watch, 2026-10-11 |
Summary
An audit by Berkeley statisticians Nguyen and Fithian pointed out that METR's 50% horizon is the output of a fitted logistic curve, not a direct measurement. The audit does not deny the exponential trend itself; it was recorded as a problem with the ruler's calibration.
A/B Impact Analysis
This item concerns the reliability of the measuring tool, not the direction of the trend, so it cannot be used directly for either scenario. It did lead to a follow-up agreement: if interpretation of the METR metric both sides use for judgment wobbles, the December 2027 crux should be judged by measured survival time instead. It is logged as neutral — a case where a methodology item changed the judging rules, not a trend-reversal signal.