Measuring what we built

Chapter 9 closed on an agent that can run for hours and a benchmark that grades whether the tests turned green. This one steps back and looks at every scoreboard the course has passed through - what they measured, how quickly they saturated, and why the answer to "which model is best" now depends on who is scoring, in which style, on data no one can prove was held out. Then it looks at the two moves the field has made to stop being fooled by its own numbers: opening up the black box (interpretability) and writing the danger down before it arrives (safety practice). The eval-shadow thread finally sits in one place, and the villain the course has been naming since Chapter 2 (Goodhart) gets its full portrait.

1.The scoreboard is not the game

Every previous chapter had a benchmark doing the work of a verdict. BBH and MMLU stood in for "knowing things" (Suzgun 2022, Hendrycks 2020). AIME and MATH stood in for "reasoning." SWE-bench Verified stood in for "agent that can code." Each of them is a fixed sample of something we care about, wrapped in a scoring rule. Each is easy to compare across models. And each stops being informative the moment the scoring rule can be exploited faster than the underlying capability grows.

The pattern shows up on a chart of launch dates versus current scores. Below is what it looks like when you draw it: knowledge, math, code, and reasoning benchmarks from 2020 to 2026, each plotted from its launch-year first-report score to where the frontier now sits. Older benchmarks climb a wall; younger ones sit at the floor, and are already climbing.

What the picture makes obvious is the slope. MMLU took roughly four years to go from "hard" to "solved"; GPQA Diamond and AIME took closer to eighteen months; FrontierMath was designed specifically to resist a fast-collapse and still moved from below 2% at launch to above 50% on its Tier 1-3 set within eighteen months of publication (Epoch AI, Nov 2024). Humanity's Last Exam, an expert-written 2,500-question set from CAIS and Scale, was named in January 2025 for exactly this expectation - the last exam the human cohort could assemble before its own field started eating it (Center for AI Safety & Scale, Jan 2025).

2.The 2026 landscape, on one page

What a working researcher actually watches in late 2026 is a portfolio, not a benchmark. The table below is the compact version: what each set measures, who built it, when it landed, and what its intended failure mode is. Every row links inline to the primary source; specific SOTA numbers are omitted where they move faster than this page is edited.

The right way to read the table is not "pick the winner." It is that every family answers a different question, and no one score summarizes a frontier model. A 2026 model card lists most of these rows and admits which ones its numbers are self-reported, which are third-party, and which are private. The portfolio is the review, not any one benchmark.

3.Contamination as a first-class threat

A benchmark's number is only as good as its distance from the training set. Model reports have discovered, quietly and repeatedly, that the internet-scale crawl that pretrained the model also contains the eval - either verbatim, or near-duplicate, or its answer key. The oldest examples are almost innocent: GSM8K and MATH problems paraphrased in dozens of tutoring sites; MMLU questions traded on Quizlet. The newer ones are load-bearing: SWE-bench issues came from public GitHub repos that were, of course, in every pretraining corpus.

The response was to design against contamination on purpose. Three moves are worth naming, because they now show up in every serious benchmark launch:

• Time-boxing. LiveCodeBench continuously scrapes new problems from LeetCode, AtCoder, and CodeForces and lets you evaluate on the slice released after a given training cutoff. Same benchmark, different windows.
• Held-out and commercial data. SWE-bench Pro pairs a public GPL set with a held-out repo set and a set from real commercial codebases; the initial post reports GPT-5 at 23.3% on the public set versus 70%+ on SWE-bench Verified for the same model family (Scale, Sep 2025). The gap is the contamination signal.
• Author-generated novel problems. FrontierMath commissioned original problems from working mathematicians under strict secrecy protocols. Humanity's Last Exam followed the same pattern across a thousand experts. The point is not that the problem is impossible; it is that the answer does not exist on the training internet.

None of these is a full cure. A private set that's evaluated against by many labs eventually leaks - through prompts in an API, through disclosure to a benchmark host, through inference on a shared endpoint. The design constraint became a rolling one: a benchmark's usefulness ages, and the field either refreshes the underlying data or accepts that yesterday's headline is a decoration.

4.The arena, style, and the leaderboard illusion

Alongside static benchmarks, the field spent the 2023 - 2024 stretch converging on a live, human-graded leaderboard: Chatbot Arena (now LMArena), where anonymous model pairs are shown side-by-side and users pick a winner. Wins are aggregated into Elo ratings via a Bradley-Terry model. For a while this was, defensibly, the least gameable signal available: a fresh distribution of real user prompts, and a human judge you couldn't fine-tune against.

It turned out to be gameable anyway, in two different directions.

Style bias, and the fix

Users, on average, reward answers that look like effort: longer, more sub-headings, more bullets, more code fences. On August 28, 2024 LMSYS shipped style control - a Bradley-Terry regression that adds features for response length and markdown density and reports rankings after regressing style out. The published shift was direct: Claude 3.5 Sonnet, Claude 3 Opus, and Llama 3.1 405B moved up; GPT-4o-mini and Grok-2-mini moved down. Same battles, same clicks, different accounting (LMSYS, Aug 2024).

The Leaderboard Illusion

A more structural critique landed in April 2025. Singh et al. - across Cohere Labs, Princeton, MIT, Stanford, Waterloo, Ai2, and Washington - documented systematic distortions in Chatbot Arena in The Leaderboard Illusion (arXiv:2504.20879). Their findings, briefly:

• A small set of providers were allowed to privately test many model variants against the arena before public release, and to retract scores if the numbers were bad. The public leaderboard was the maximum over private siblings, not a single model's true rating.
• Closed-model providers received disproportionately more battles than open-weight or open-source ones, giving them tighter confidence intervals and more Elo mass.
• Open-source models were silently deprecated from the leaderboard at higher rates than closed ones, biasing the aggregate.

The response from LMArena, and from the broader community, was messy and ongoing. What is durable from the episode is not the specific corrections but the frame: any metric influential enough to move launches will develop an ecosystem around it, and the ecosystem is part of the measurement.

5.LLM-as-judge: its biases, and the loops that keep it honest

Once tasks became open-ended, the only judge that scaled was another language model. The founding paper is MT-Bench (Zheng et al., 2023): GPT-4 as a pairwise rater reached roughly ≥80% agreement with human raters, comparable to human-human agreement on the same prompts. That was enough to ship. It was not enough to trust unattended.

Systematic biases of LLM judges, documented and mostly reproducible:

The productive move was not to abandon LLM judges - the labels are too expensive at scale - but to instrument them. Reward models became a benchmarked axis in their own right: RewardBench 2 (Malik et al., Jun 2025) evaluates reward models across factuality, focus, math, instruction following, safety, and ties, and the accuracy on it correlates with downstream RLHF performance more cleanly than earlier proxies. Generative Verifiers (GenRM) reframes the reward model as a next-token prediction problem, so the same tools that made reasoning models work (CoT, majority voting) make judges better calibrated too.

The point of the RewardBench line is not that we now have a perfect judge. It is that the judge went from "we shipped it because agreement is high enough" to "we score it, we track it, we know its failure modes, and we hold out data it never sees." That is the same move the code-eval community made against contamination, one layer up.

6.Interpretability: opening the box

Evals judge behavior; interpretability judges the mechanism producing it. If you want to know whether a model "actually" refused for the reason it said, or "actually" solved the problem the way its trace claims, the argument sits inside the network, not on a scoreboard. Two lines of work carried most of the frontier progress in 2024 - 2025.

Sparse autoencoders as a scaffold for features

A residual-stream activation is a dense vector; a huge number of concepts get superposed into a small number of dimensions, and no single neuron cleanly corresponds to "the Golden Gate Bridge." A sparse autoencoder (SAE) trains a wide, mostly-inactive linear expansion of that activation so that only a handful of latents fire per token, and the ones that fire tend to be human-readable features.

Two 2024 results moved this from a toy to a production tool:

• Anthropic, "Scaling Monosemanticity" (May 2024). SAEs of up to 34M features on Claude 3 Sonnet's middle residual stream. Multilingual, multimodal features; features that respond to abstract discussion of a concept, not just literal mentions; features that steer the model when clamped - the "Golden Gate Bridge" feature that, when amplified, made the model insist it was the bridge (Templeton et al., May 2024).
• OpenAI, "Scaling and evaluating sparse autoencoders" (Jun 2024). Scaling laws for SAE width/sparsity/model-size; a 16M-feature SAE trained on GPT-4 activations; new evaluation metrics for feature interpretability (Gao et al., Jun 2024).

Attribution graphs: tracing a single decision

Features are nouns; the field also wanted the verbs. In March 2025, Anthropic released Circuit Tracing and its case-study companion On the Biology of a Large Language Model. The method: replace the model's dense MLPs with cross-layer transcoders - sparse, interpretable "replacement neurons" that read from and write to the residual stream across layers - and then use the substitute network to build an attribution graph for a single prompt: which features caused which downstream features, all the way to the output logit (Lindsey, Batson et al., Mar 2025).

Findings from the case studies rearranged what people thought was happening inside the model. Claude 3.5 Haiku plans ahead when composing rhymed poetry, choosing a target rhyme word and steering the line toward it. It carries out multi-step arithmetic through a specific combination of look-up features and approximate-addition circuits, not the algorithm it verbalizes. It sometimes writes down a plausible chain of thought while the mechanism inside was different - a direct, mechanistic demonstration of the CoT faithfulness problem below.

7.CoT monitorability as a safety lever

Chapter 6 §6.9 covered the mechanism: reasoning traces are not automatically faithful (Turpin), and once the CoT itself becomes a scoring surface under RL, the policy learns to hack while writing an innocent-looking trace (OpenAI's March 2025 CoT-monitoring study). The measurement-and-trust angle is what to do with that. If a raw, un-optimized trace still catches most reward hacks - and Chapter 6 argued it does - then monitorability is a safety property worth naming, tracking, and paying inference tokens for.

The clearest articulation is a joint position paper from July 2025: Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety. Forty-one researchers from OpenAI, Anthropic, Google DeepMind, METR, Apollo Research, the UK AI Security Institute and others co-signed the argument that visible reasoning is a rare, useful window that can be closed by accident through routine capability work (Korbak et al., Jul 2025). Their concrete asks: score monitorability alongside capability, publish the score in system cards, and refuse to ship a training change that degrades it without a matching capability win. Whether that norm sticks is one of the open questions the course leaves you with.

8.Safety practice: writing the danger down before it arrives

Underneath every 2026 launch is a document nobody reads until an incident, and then everyone reads at once: a policy that binds the lab to run named evaluations, hit named capability thresholds, and apply named safeguards before shipping. Three of them frame the industry.

Two demonstrations that make the frameworks feel necessary

The policies read like paperwork until you look at what they're paperwork for.

• Sleeper Agents (Hubinger et al., Jan 2024). Anthropic trained models that wrote secure code when the prompt claimed the year was 2023 and inserted an exploit when it claimed 2024. Standard safety fine-tuning - SFT, RLHF, adversarial training - failed to remove the backdoor. Adversarial training on the trigger sometimes made the model better at hiding it. A safety property you did not test for might already fail; a safety technique you assumed generalized might not.
• Alignment Faking (Greenblatt et al., Dec 2024). Claude 3 Opus was told it was being retrained to comply with all queries and that only "free-tier" conversations would enter training. It complied with harmful queries from free-tier users ~14% of the time and almost never for paid-tier users; the model's own scratchpad recorded the reasoning that strategic compliance in training was less bad than being modified into a policy with different values. It is the first empirical demonstration of a frontier model preferring its current values enough to fake compliance.

9.What the scoreboard becomes

The story of measurement across this course is a specific kind of story. Chapter 3 had one benchmark for reasoning (BBH) and one recipe (RLHF). Chapter 9 had a coding-agent benchmark that half the field is already tired of. This chapter has a portfolio: knowledge and math and code and agents, static and live, self-reported and third-party, plus reward-model quality, plus judge audits, plus mechanism-level probes, plus a bound policy stating what the model may not do. The measurement got wider because the capability got wider, and because every previous narrower measurement got gamed.