Build-A-Chip-Bench — best overall score only 16.6% despite up to 72 hours per task
| Date | 2026-10-14 |
|---|---|
| Category | Benchmarks |
| Judgement | Refutes |
| Gauge weight | 2 / 3 |
| Source | ChipAgents and NVIDIA Build-A-Chip-Bench (BusinessWire, 2026-10-14), recorded in AGI watch evening run |
Summary
On Build-A-Chip-Bench, which measures the full spec-to-GDS flow, the best overall score was 16.6% by Claude Fable 5.1. GPT-6 Astra passed function and GDS at 85% but scored 10.2% on PPA, for an overall 8.7%.
A/B Impact Analysis
With physical gates that cannot be gamed, this benchmark exposes a pure capability gap, which is Scenario B evidence. The failure modes — destroying verified behaviour while optimising, and slow backtracking from early decisions — concretely show the bottleneck in long-horizon autonomy. For Scenario A, the benchmark's existence signals rising measurement pressure, but the scores themselves cannot be used as support. It is recorded alongside the hold on the median.