AGI - Are we there yet?

Real-time interactive environments generated from a prompt, holding physical and visual consistency over minutes. World models are the missing training ground for embodied agents, and they got materially better.

Two labs independently reached gold-medal scores on the same contest, working in natural language within the official time limit. Formal, closed-domain reasoning is close to solved; the open-ended version is not.

Sustained autonomous engineering sessions moved from demo to shipped capability, with customers reporting hours of unattended work. This is the dimension where the gap between benchmark and billable output is narrowest.

The best single source for the base rates behind this page: benchmark movement, cost-per-token collapse, the narrowing gap between open and closed models, and the fact that responsible-AI evaluation is not keeping pace with capability.

The counterweight to the entry above. A successor benchmark, still easy for humans, dropped frontier scores back toward the floor — a reminder that saturating a test is not the same as acquiring the capability it was built to measure.

The single most load-bearing measurement on this page. The length of task a model finishes at a 50% success rate has been doubling roughly every seven months — which is the strongest reason to think the autonomy score keeps climbing, and the clearest evidence of how low it still is.

Measured usage, not projected usage. AI is concentrated in a narrow slice of occupations and skews toward augmenting a person rather than completing the task alone — which is why economic substitution stays the lowest-scoring dimension we track.

Commissioned because the previous generation of academic benchmarks had been saturated. Frontier scores started in single digits and have climbed steeply since — worth watching precisely because it was designed to be the last hard exam.

A benchmark built explicitly to resist memorisation went from near-zero to human-comparable in a single model generation, at very high inference cost. Test-time compute became a real axis of capability, not a research curiosity.

Problems written by working mathematicians and never published, so they cannot be in the training data. Progress here is the cleanest available proxy for genuine mathematical reasoning as opposed to recall.

A single generalist policy driving multiple robot platforms across dextrous tasks. Robotics is now on the foundation-model curve — a few years behind language, and bottlenecked on data rather than on architecture.

Opened up every task that has a GUI but no API. Early accuracy on realistic desktop tasks was far below human, which is exactly why reliability — not capability — is the dimension gating agent deployment.