METR's chart has an axis nobody used to plot: how long a task an AI agent can finish on its own. The doubling time is 7 months, traced from GPT-2 through Claude 3.7 Sonnet and measured in human work-minutes. Then the same lecture puts up the 80%-reliability version of that chart, and the horizon collapses. Both lines are real. The distance between them is where most of the hard problems in agent research live, and it turned out to be the through-line across all 9 lectures.
Last month I spent 3 weeks working through the lectures Stanford Online published from CS 329A: Self-Improving AI Agents, taught by Azalia Mirhoseini (Assistant Professor of Computer Science at Stanford, where she directs the Scaling Intelligence Lab) and Aakanksha Chowdhery (Adjunct Professor at Stanford, previously technical lead of the 540B PaLM model at Google). This post walks through them in the order they were taught: course overview, test-time compute scaling, robust verification, learning from feedback with tools and code, planning and multi-step reasoning, train-time scaling and RL, search and deep research agents, agentic evals and long-horizon tasks, and future research areas. The course site is public and lists every paper reading per lecture, which is worth the bookmark even if you never press play.
So What Makes An Agent Self-Improving?
Every lecture in the course is a variation on one loop. Generate candidate solutions or actions. Verify them against something: a unit test, a reward model, another LLM, a person. Select what survives, or train on it. Then run it again. Lecture 1 breaks an agentic workflow into its parts: LLM calls, tool calls, verifiers and critics, and orchestration patterns like prompt chaining and routing, all held together by predefined code. Self-improvement is what happens when the output of that loop feeds back into the next turn of it, at inference time or in the weights.
The instructors name the generation-verification gap in lecture 1, before they teach a single technique. That ordering is the argument. Sampling a model 250 times raises the odds that a correct answer sits somewhere in the pile, and does nothing to tell you which one it is. Reinforcement learning needs a reward, and a reward needs something that can score an answer faster and cheaper than a person can. Coding got there first because a unit test is free. The generator has raced ahead of the verifier. Nearly everything that follows in this post (planning, deep research agents, long-horizon evals) is a story about what happens when verification isn't free.
1 — What A Self-Improving Agent Actually Is
The first lecture is a history lesson with a turn at the end of it. It runs in three moves, and the third one is the whole course.
The first move is train-time scaling. Scaling laws turned more compute into more capability on a predictable curve, and along the way they bought things nobody trained for: few-shot and zero-shot performance on tasks that never appeared in the data, and chain-of-thought reasoning arriving as emergent behavior rather than as a feature someone shipped. But ChatGPT launched on an instruction-tuned GPT-3.5, and the instructors are precise about which part did which work. Scale produced a model that predicts the next token well. The alignment stack produced one people wanted to talk to, and the instructors lay it out in three steps:
Next-token prediction is the pre-training objective. It yields a model that knows an enormous amount and follows nothing.
Instruction tuning is fine-tuning across many tasks, so the model follows an explicit instruction and answers in the format you asked for.
RLHF has humans rank outputs on correctness, helpfulness, and specificity. A reward model is fit to those rankings so it can stand in for the humans, and a KL-divergence penalty keeps the tuned policy from drifting too far from the model it started as.
The second move is the pivot to inference. Large Language Monkeys (Brown et al., 2024) is the paper the lecture uses to make the point, and the point is almost rude in its simplicity: ask the same model the same question enough times and it gets dramatically better. The lecture's example is LLaMA-3-8B clearing GPT-4o on reasoning benchmarks: an 8B open model past a frontier one, with no new training, just more attempts.
The third move is reasoning models. DeepSeek-R1, OpenAI's o-series, Gemini Flash-Thinking: test-time scaling first, then distilling the synthetic reasoning traces back into the weights, which closes the loop between the two scaling axes. Here is what a chain of thought actually contains, read as a procedure instead of as text: analyze the problem, decompose it into tasks, evaluate its own progress, correct itself, and propose an alternative when the first route dead-ends.
Which brings the lecture to its definition. An agent is a system where the LLM directs its own process and its own tool use, keeping control over how the task gets done. An agentic workflow is LLM calls and tools orchestrated through predefined code. That distinction sounds academic and isn't: an agent is not a workflow with a model dropped into it; it's a system that chooses its own next step. Coding agents are the version that works today: LLM calls, computer-use actions, and feedback from what happened, pointed at repetitive work. Customer support agents and research agents like Sakana AI's AI Scientist (Lu et al., 2024) are the same shape with a worse verifier.
One thing from this lecture stayed with me through the other 8. Self-evaluation and self-correction are listed here, in the first hour, as capabilities reasoning models already have.
2 — Test-Time Compute And The Monkey Problem
Gemma-2B gets 0.02% of CodeContests problems right on its first attempt. Sample it enough times and coverage climbs by over 300x. That is the result Large Language Monkeys (Brown et al., 2024) is named for, and it holds across model sizes from 70M to 70B and across the Llama, Gemma, and Pythia families. Coverage (the fraction of problems solved by any generated sample) scales with the number of samples over four orders of magnitude, log-linear, fitted with an exponentiated power law.
Coverage is not the same as getting the answer, and the gap between those two things is what the rest of the course is about. On SWE-bench Lite, DeepSeek-Coder-V2-Instruct goes from 15.9% of issues resolved with 1 sample to 56% with 250, past a single-sample state of the art of 43%. That works because a test suite tells you which of the 250 was right. Where no automatic verifier exists, majority voting and reward-model scoring flatten out past a few hundred samples. The pile keeps getting better. Your ability to reach into it does not.
The follow-up paper is the one I enjoyed most. How Do Large Language Monkeys Get Their Power (Laws)? opens on a contradiction: on any single problem, a simple calculation says the failure rate should fall exponentially with more attempts, and empirically it does. So why is the aggregate curve a power law? Because the distribution of single-attempt success probabilities is heavy-tailed. A small set of problems the model almost never solves warps the aggregate away from exponential, even while every individual problem still scales exponentially on its own. The long tail of hard problems is what makes the curve a power law.
Snell et al. (2024) asks the budgeting question instead. Given a fixed amount of inference compute, do you spend it on parallel samples or on sequential revisions of a single answer? Which one wins depends on how hard the prompt is, which is why the paper lands on a compute-optimal strategy that allocates adaptively per prompt rather than picking a side. Two selection mechanisms sit underneath it: outcome reward models score the finished answer, and process reward models score each step, which lets you run beam search through the reasoning instead of only ranking completed attempts. Allocating adaptively improves test-time scaling efficiency by more than 4x over a best-of-N baseline, and in a FLOPs-matched comparison, test-time compute beats a 14x larger model; on problems where the smaller model already had a non-trivial success rate. That last clause is the whole caveat.
Archon (Saad-Falcon et al., 2024) treats the inference-time stack as something to search rather than something to hand-design. Generation, ranking, fusion, critique, verification, and unit testing become layers you can stack, and Bayesian optimization picks the arrangement for a given compute budget and set of available models. Two things stood out to me. Deep architectures, with several layers of critique and fusion, beat spending the same budget on more samples from the single best model. And natural-language unit tests give you a verifier for tasks where no real test suite could exist. Archon's searched architectures outperform frontier models including OpenAI's o1, GPT-4o, and Claude 3.5 Sonnet by an average of 15.1%.
One thing I caught only on the third paper: Mirhoseini is an author on 3 of the 4 readings for this lecture, which makes it less a survey than a tour of her lab's argument. Inference compute converts into capability predictably, and your verifier sets the exchange rate. The lecture closes by naming the limit rather than solving it. KernelBench has repeated sampling working for CUDA generation, and in nearly the same breath the instructors say existing verification methods need drastic improvement, because on the hardest problems a correct generation is rare enough that finding it is the entire job.
3 — Verification Is The Bottleneck
The idea is older than the current wave of interest in it. In 2021, Cobbe et al. released GSM8K (8.5K grade-school math word problems) largely because no good benchmark existed for multi-step arithmetic reasoning, and then did the part that mattered more: they trained a verifier. Generate many candidate solutions, score each one for correctness, keep whichever ranks highest. The verifier is itself a language model with a small scalar head predicting, token by token, the probability that a solution ends up right, trained on a joint objective alongside ordinary language modeling. Their headline finding has held up for 5 years: verification scales better with added data than fine-tuning does.
Let's Verify Step by Step (Lightman et al., 2023) asks what the verifier should be looking at. An outcome reward model scores the finished answer. A process reward model scores every intermediate step. The paper's justification is the line I wrote down twice: a single logical error is enough to derail a much larger solution. Process supervision wins on three counts. It puts credit where the mistake actually was, it declines to reward a right answer reached through wrong reasoning, and it trains the behavior you want instead of the outcome you can measure. OpenAI collected 800,000 step-level human labels across 12,000 problems to build PRM800K, and the process-supervised model solves 78% of a representative subset of the MATH test set. Active learning stretched those labels further by surfacing the solutions the current model rates highly and still gets wrong.
800,000 human labels is a strong result and a terrible business. Math-Shepherd (Wang et al., 2023) removes the humans, and its definition of a good reasoning step is operational rather than semantic: a step is good to the degree that continuing from it can still reach the correct final answer. You estimate that by rolling forward from the step many times and counting. Hard labels mark a step correct if any continuation lands right; soft labels use the fraction that do. Trained on those annotations, step-by-step PPO takes Mistral-7B from 77.9% to 84.1% on GSM8K and from 28.6% to 33.0% on MATH, and adding Math-Shepherd's verification on top pushes the same model to 89.1% and 43.5%. The author list includes DeepSeek-AI, which is worth holding onto until lecture 6.
Weaver (Saad-Falcon et al.) goes at the gap directly, and its method is three words: score, weight, select. Run every candidate past many imperfect verifiers: LM judges, reward models, whatever is available. Weight each verifier by how accurate it actually is. Select on the combined score. The obstacle is that you have no labels to learn those weights from, so Weaver borrows weak supervision to estimate each verifier's accuracy from agreement patterns, normalizes inconsistent output formats using dataset statistics, and filters out the verifiers that are only adding noise. Weighted ensembles beat unweighted ones, which is the entire reason the machinery exists.
The numbers are what make this lecture land. Weaver shrinks a model's generation-verification gap by 14.5% on average. Llama 3.3 70B Instruct (a cheaper, non-reasoning generator) paired with an ensemble of 70B-or-smaller judges and reward models reaches o3-mini-level accuracy at 87.7% average. That is roughly the size of the jump from GPT-4o to o3-mini (69.0% to 86.7%), except o3-mini got there through extensive finetuning and post-training, and Weaver got there by choosing better from what the generator already produced. Running an ensemble of 70B verifiers over every candidate is untenable in production, so they distill the ensemble into a 400M cross-encoder that retains 98.7% of Weaver's accuracy while cutting verification compute by up to 99.97%.
Here is the thing I kept circling. Every benchmark in this lecture (GSM8K, MATH, GPQA Diamond, MMLU-Pro) has an answer you can check by comparing strings. The methods are general in principle. All of the evidence was earned in the one place where the ground truth was already sitting there.
4 — Feedback From Tools, Code, And A Constitution
The ReAct paper (Yao et al., 2022) opens in a kitchen. You are cooking, and between two physical actions you think in language: now that everything is cut, heat the pot of water. I don't have salt, so soy sauce and pepper instead. The paper's argument is that reasoning and acting had been studied as separate problems, and that interleaving them is what makes either one work. The reasoning trace tracks the plan, updates it, and handles exceptions. The actions go fetch what the model doesn't have.
Then the results. On HotpotQA and Fever, ReAct with access to a simple Wikipedia API cuts through the hallucination and error propagation that plain chain-of-thought runs into, because the model can go check. On ALFWorld and WebShop it beats imitation-learning and RL baselines by an absolute 34% and 10% in success rate, prompted with only one or two in-context examples, and its best ALFWorld trial averages 71%. The traces are legible to a person, which the paper counts as a result rather than a nice side effect. I'd agree: you can read what the model believed at step 4 and why it acted the way it did at step 5.
The weaknesses are the ones you would guess. A large action space needs more demonstrations than fit in the context window. Interleaving reasoning with every action costs tokens and latency against a model that simply acts.
RLEF moves the loop from the prompt into the weights. RLEF trains code models end-to-end to use execution feedback: generate a program, run it against public tests, read what broke, revise, repeat up to a turn limit. Private tests supply the reward, which stops the model from memorizing its way to a pass. The policy update is a hybrid (token-level generation, turn-level evaluation) and the two-tier test split is doing the quiet work underneath it. Run the code. Read the error. Fix it. Try again.
The result reads as a direct reply to lecture 2. Repeated sampling bought a state of the art by drawing 250 samples; RLEF reaches new state-of-the-art results on competitive programming with both 8B and 70B models while cutting the samples required by an order of magnitude. Teaching a model to read a stack trace is cheaper than sampling until one of its guesses happens to pass. The caveats are real, though. RLEF is tuned for code synthesis and competitive programming, it depends completely on execution feedback being available and informative, single-turn performance gets worse under the RL training, and the gains flatten out past about 5 turns.
Constitutional AI asks what you do when there is no test suite at all. Anthropic's answer is to write the rules down (16 principles, in the lecture's telling) and have the model apply them to itself. The supervised phase samples from the initial model, generates self-critiques and revisions against those principles, and finetunes on the revised responses. The RL phase has a model judge which of two samples is better, trains a preference model on that dataset of AI preferences, and runs RL with it as the reward signal. That is RLAIF, and the claim in the abstract is the one that stops you: no human labels identifying harmful outputs anywhere in the process. The only human oversight is the list.
What comes out is an assistant that engages with a harmful query by explaining its objections instead of deflecting: harmless without being evasive. What also comes out, and the lecture is straight about it, is a tradeoff. Harmlessness improves with each round of revision while helpfulness declines.
Notice what happens across the 3 papers. The feedback gets cheaper and vaguer at every step: an environment you can query, then a test suite that returns pass or fail, then a document somebody wrote. The document is the one I keep thinking about. Someone has to draft those principles, decide what happens when two of them conflict, and keep them current as the product changes.
5 — Planning As A Control Problem
LATS (Zhou et al., 2023) posts 92.7% pass@1 on HumanEval with GPT-4 and never updates a weight. It gets there by refusing to commit to its first plan. Generate N candidate action sequences, run them, score each one using the model's own self-reflection, then keep exploring from whichever state scored best. Monte Carlo Tree Search does the bookkeeping, LM-powered value functions do the scoring, and an actual environment supplies external feedback so the scores are not purely self-graded.
The framing from the lecture stuck with me because it makes the paper legible in one line. LATS is chain-of-thought for decomposing the reasoning, plus tree search as the planning algorithm, plus ReAct for folding environment feedback back in. Every piece had already appeared in the course. The contribution is the assembly. On WebShop, a benchmark built from 1.18 million real products and 12,000 human instructions, LATS reaches an average score of 75.9 with GPT-3.5: gradient-free, and comparable to gradient-based fine-tuning.
Two costs ride along, and the lecture hides neither. A tree search over LLM calls is expensive in exactly the way lecture 2 warned about. And the apparatus assumes an environment you can query cheaply and back out of, which a web store simulator gives you and a production database does not.
SPRINT changes the question from what to try next to what can run at the same time. Two roles: a planner, and a pool of executors carrying out independent steps concurrently. The interesting part is the data pipeline, because the parallelism gets taught rather than hardcoded. Take a reasoning model's own thinking trajectories, extract the steps, build a DAG of execution dependencies, pack the independent steps into compact stages, and fine-tune the model on that reformatted data. A small amount of curated data turns out to be enough.
The results are reported in sequential tokens rather than accuracy, which is the honest unit for this claim. SPRINT matches reasoning-model performance on math while generating up to 39% fewer sequential tokens on problems needing more than 8,000 output tokens, and the effect transfers out of distribution: up to 45% fewer on GPQA and 65% fewer on Countdown for longer trajectories. The parallelization pattern it learns is the one you would want. Harder problems get more parallelism early, exploring several approaches before converging on a reliable one.
SWiRL (Goldie, Mirhoseini, et al.) takes the part nobody enjoys: credit assignment across a long trajectory. A tool-using agent has to do four things. Know when to call a tool. Write a query worth sending. Stay accurate across every remaining step, or recover when it doesn't. Know when it has enough to stop and answer. Traditional RLHF treats all of that as a single step with a single reward at the end.
SWiRL breaks each multi-step trajectory into sub-trajectories, one per action the original model took, filters them, and runs RL on the pieces. Relative accuracy improves by 21.5% on GSM8K, 12.3% on HotPotQA, 14.8% on CofCA, 11.1% on MuSiQue, and 15.3% on BeerQA. The transfer result is the one I'd quote at people: training only on HotPotQA (text question-answering) improves zero-shot performance on GSM8K, a math dataset, by a relative 16.9%. The lecture adds that 1,000 synthetic trajectories were enough to produce real gains, and that models learn best from process-filtered data rather than outcome-filtered. That is lecture 3's argument again, in a different setting.
One number in this lecture is doing less work than it appears to. SPRINT's 39% counts sequential decoding steps, not seconds, and the paper lists realizing the wall-clock speedups as future work. Shortening the critical path on paper and shortening it on a machine with real schedulers, real memory pressure, and real tool latency are separate problems.
6 — Train-Time Scaling And What RL Actually Buys
STaR (Zelikman et al., 2022) has one move I find genuinely clever. Prompt the model with a handful of rationale examples, let it generate reasoning across a large set of unlabeled questions, and fine-tune on whichever rationales produced the right answer. When it gets one wrong, hand it the correct answer and ask it to produce the reasoning that would have led there. The paper calls this rationalization, and it manufactures training data for exactly the problems the model couldn't solve, which is where the data is scarcest.
The loop is generate, filter on correctness, fine-tune, repeat. STaR performs comparably to fine-tuning a model 30x its size on CommonsenseQA, with the human out of the loop after those first few examples. The failure mode is built into the filter. Keeping only rationales that reached a correct answer throws away sound reasoning that slipped at the last step and keeps unsound reasoning that happened to land. Remember that second one.
DeepSeekMath is a data paper and an algorithm paper sharing one title. On the data side, DeepSeek continued pretraining a 7B coder model on 120B math-related tokens mined from Common Crawl, and the lecture's point is that curation beat prestige: the pipeline starts from OpenWebMath, trains a fastText classifier, mines Common Crawl, and iterates into new math domains. Training on ArXiv hurt. Code-then-math ordering helped. DeepSeekMath 7B scores 51.7% on the competition-level MATH benchmark with no external toolkits and no voting, approaching Gemini-Ultra and GPT-4, and self-consistency over 64 samples takes it to 60.9%.
On the algorithm side, GRPO drops the critic. PPO carries a policy, a reference policy, a critic, and a reward model, and the memory bill is brutal. GRPO samples a group of answers to the same question, normalizes the rewards inside that group, and uses the group as its own baseline. No critic, roughly half the memory, and it works because reward models were trained on comparisons in the first place.
Here is where I want to push back on how this era gets narrated. The standard story is that RL on verifiable rewards teaches models new capabilities, that o1 and R1 and their descendants can do things their base checkpoints fundamentally could not. The story isn't baseless. RL-trained models visibly solve problems their starting checkpoints failed.
DeepSeekMath ran the measurement anyway, in a section titled "Why RL Works?". RL improves Maj@K and does not improve Pass@K. Sample the model 32 times, as the lecture's chart does, and the odds that at least one answer is correct barely move; what moves is how often the majority lands on the right one. The paper's own reading is that the improvement "is attributed to boosting the correct response from TopK rather than the enhancement of fundamental capabilities." The instructor said it more bluntly in lecture: the model became more consistent, not fundamentally smarter.
That distinction carries further than it usually gets credit for, because a selection gain is only as portable as the thing doing the selecting. In math and code, where a checker is free, RL converts coverage into accuracy and the result reads like new capability. Carry the same method into research synthesis or scientific discovery, where no cheap checker exists, and there is nothing to convert with. STaR's false positives are the same phenomenon showing up at training time: filter on outcomes and you reward the lucky guess alongside the sound one. Lectures 7 through 9 spend most of their time walking into this wall.
DAPO is the part of this lecture I'd hand to anyone actually running these jobs. ByteDance Seed and Tsinghua AIR open-sourced an entire RL system (the algorithm, the training code on verl, and a curated dataset) explicitly because o1's blog post and R1's technical report withheld the details the community needed to reproduce them. It reaches 50 points on AIME 2024 with a Qwen2.5-32B base model, against the 47 the lecture's slide credits to DeepSeek-R1-Zero-Qwen-32B. Clip-Higher, which permits larger upward moves in probability than downward ones, is shown lifting AIME from 30% to 38% on its own.
The most useful slide in the whole lecture is the operational one. Don't watch the loss. Watch response length, entropy, and the share of samples scoring exactly 1. Length suddenly exploding means go check your token-level loss. Entropy sliding to 0.2 is entropy collapse, and exploration is gone. No improvement after 2,000 steps and the reward model may be saturated. Dynamic sampling exists for a reason the instructor stated flatly: if there is not a distribution of rewards, there is nothing for the model to learn. A batch of all zeros and a batch of all ones normalize to the same nothing.
The open-problems slide closing this lecture leads with the question this section has been circling: why Maj@K but not Pass@K? It sits there unanswered. The second item is "are emergent behaviors real?", which is a striking thing to find on a slide in week 6 of a graduate course largely built on the assumption that they are.
7 — Search At Scale And Deep Research Agents
One million samples per problem. That is the number AlphaCode (Li et al., 2022) generates before it submits anything, and the pipeline around that number is the interesting part. Pretrain an encoder-decoder on GitHub, with a masked language modeling loss on the encoder and ordinary next-token cross-entropy on the decoder. Fine-tune with tempering to sharpen the training distribution, value conditioning so the model sees whether a submission was correct, and GOLD to concentrate learning on high-likelihood tokens. Then sample a million programs, filter down to the ones that pass the example tests printed in the problem statement, and cluster what survives so that syntactically different but semantically identical programs collapse into one candidate. Ten submissions go in. AlphaCode lands at a top 54.3% ranking averaged over 10 Codeforces contests with more than 5,000 participants each.
AlphaCode 2 keeps the shape and replaces nearly every component. Fine-tune Gemini Pro rather than pretrain on GitHub. Generate a family of fine-tuned models with varied hyperparameters, because diversity in the candidate pool is what the search is actually feeding on. Add CodeContests V2 and a second, higher-quality dataset. And then the piece that matters most: fine-tune another Gemini Pro model whose only job is to score how likely a candidate is to be correct, before anything gets submitted.
The results land where you'd hope. AlphaCode 2 solves 43% of problems within 10 attempts against AlphaCode's 25%, and sits at the 85th percentile on average where its predecessor beat an estimated 46% of competitors. In the two contests where it does best, it outperforms more than 99.5% of the field. But the line I underlined is about efficiency: AlphaCode 2 needs roughly 100 samples to reach the performance AlphaCode got from a million. A million samples. Then a hundred. Same contests.
Which is the argument from lecture 6 showing up as an engineering result. Nothing about competitive programming got easier between the two systems. The generator improved, and the thing choosing among its outputs improved a great deal more: a learned correctness scorer, a deliberately diversified candidate pool, and a filter grounded in tests that came with the problem. This is what a selection gain looks like when you have a free verifier. The weaknesses the lecture lists are the ones you'd expect: the experimentation is expensive to run at this scale, and all of it is very specific to code.
Search-o1 is what the same instinct looks like outside code, and it starts from a failure I hadn't thought to measure. Large reasoning models hedge. The paper analyzes QwQ-32B-Preview's traces and counts the uncertain words ("perhaps," "maybe," "alternatively") that show up when a long chain of thought runs into something the model doesn't know. One example from the paper: "Wait, perhaps it's referring to dimethyl sulfone." The chain keeps going from there, and the gap propagates.
Search-o1's answer is retrieval that the model triggers itself. When reasoning hits an uncertain knowledge point, it searches, rather than retrieving everything once up front and hoping the right document is in there. The other half is a Reason-in-Documents module, and this is the design choice I'd steal. Retrieved documents are long and noisy, so a separate pass analyzes them against the current query and the reasoning so far, and emits only refined information back into the chain. Raw documents never touch the reasoning trace, because dropping them in whole breaks the coherence the model spent thousands of tokens building. Simple retrieval makes the context longer. Integration is the actual problem.
There's a limit under all of this that the lecture names without solving. Every system here searches the space of things the model can already produce. AlphaCode's million samples work because a correct program is somewhere in that pile, and Search-o1 works because the missing fact is somewhere on the web. Neither one tells you what to do when the answer isn't in the space you're searching.
8 — Measuring Agents Over Long Horizons
o3 has a 50% time horizon of about 110 minutes. That is METR's metric and it deserves the space it takes to explain: the length of task, measured in how long a skilled human needs for it, that a model completes with 50% success. Human time-to-complete becomes the universal ruler, which is what makes the number legible outside of AI. 50% is chosen because it sits between capability and reliability: high enough to mean something, low enough that the measurement doesn't collapse into noise.
The task suite spans 170 tasks across 6 orders of magnitude of duration, assembled from RE-Bench, HCAST, and 66 shorter tasks built for the paper, with domain experts timed on their successful attempts. Frontier time horizon has doubled roughly every 7 months since 2019, and the paper notes the trend may have accelerated since 2024. What's driving it, per the abstract, is greater reliability, better adaptation to mistakes, stronger logical reasoning, and better tool use. Extrapolate and you get the paper's own forecast: within 5 years, models automating many software tasks that currently take a human a month.
The failure modes are worth reading next to that forecast. Poor planning and tool choice. Mental math that goes wrong. Abandoning a task early. Trying the same failed action again. Two caveats sit in the paper and not in most of the coverage: the title says software tasks, and timing humans only on their successful attempts may understate how hard the tasks really are.
GDPval measures something else entirely. OpenAI built it from the actual work product of professionals averaging 14 years of experience, covering 44 occupations across the top 9 sectors of U.S. GDP, at least 30 tasks per occupation, which puts the full set around 1,320 tasks. The deliverables are predominantly digital. Grading is head-to-head comparison against a human expert rather than a scored answer key, because these tasks don't have one; and that choice is the whole reason the benchmark exists. A gold subset of 220 tasks is open-sourced with an automated grading service.
The experiment I keep thinking about is the context ablation. Strip context out of the prompts and model performance falls, because models struggle to work out what they should be working on. Real work is mostly that. The brief arrives underspecified, and the first hour goes to figuring out what the request actually is. The paper also reports that more reasoning effort, more task context, and more scaffolding each improve results, which is a polite way of saying the benchmark rewards you for handing the model the context a colleague would have gone and found.
Here is the tension the lecture builds and then leaves standing. METR's curve is exponential, doubling every 7 months. GDPval's is roughly linear, with frontier models approaching expert deliverable quality at a steady walk. Both are measuring agents. Both are careful papers. They disagree about the shape of the thing, and which curve you cite determines almost everything you'd conclude about the next 3 years.
DeepScholar-Bench, from a Stanford and Berkeley group that includes Ion Stoica and Matei Zaharia, takes a third angle: measure the quality of long-form synthesis directly. The task is generating a related work section for a recent arXiv paper, which requires retrieving prior work, reading it, synthesizing it, and citing it correctly. The benchmark is live, drawing fresh queries and human-written exemplars from new papers, which is how it sidesteps the contamination and staleness that eventually ruin any fixed test set. Performance is scored on knowledge synthesis, retrieval quality, and verifiability. No system in their evaluation (open-source pipelines, search agents on strong models, OpenAI's DeepResearch) surpasses a geometric mean of 31% across all metrics.
The lecture closes on a list of things the instructors hold with low confidence, and it's the most honest slide in the course. Highly context-dependent work. Navigating ambiguity and working out the requirements. Adversarial environments. Long-term reliability above 95% success. Generalization past software and knowledge work. Reaching a time horizon and producing good work at that horizon are different achievements, and only one of them has a doubling time.
9 — What Is Still Open
Gold-level scores on IMO 2025. Gold on CMO 2024. A near-perfect 118 out of 120 on Putnam 2024 with scaled test-time compute. That's DeepSeekMath-V2, and the reason it opens this section rather than sitting in lecture 6 is the sentence in its abstract: correct answers don't guarantee correct reasoning.
DeepSeek is blunt about why final-answer RL ran out of room. Rewarding a model for matching ground-truth answers took LLMs from poor to saturating AIME and HMMT inside a year, and it has two limits. A model can reach the right answer through flawed logic or fortunate errors, which makes the reward an unreliable proxy for the reasoning. And a large share of mathematics (theorem proving above all) has no numerical answer to check at all. So they went at the verifier directly: train an accurate and faithful LLM-based verifier for proofs, use it as the reward model for a proof generator, and incentivize the generator to find and resolve as many issues in its own proofs as it can before finalizing them. A meta-verifier sits on top to catch hallucinated issues, because a verifier that invents problems is its own failure mode.
The part I'd underline is how they keep the verifier ahead. As the generator improves, it produces proofs the verifier struggles to judge, so they scale verification compute to automatically label those hard cases and train the verifier on them. The generation-verification gap stops being something you suffer and becomes something you manage. Every earlier lecture in this course treats verification as the constraint. This one treats it as the product.
Absolute Zero goes after the other bottleneck. Even RLVR in the so-called zero setting still needs a human-curated pile of questions and answers, and the paper names the long-term problem plainly: high-quality human examples are scarce, and in a future where models surpass human expertise, human-set tasks stop carrying much to learn. Their answer is a single model that proposes tasks maximizing its own learning progress and then solves them. The Absolute Zero Reasoner generates code-based tasks in 3 modes (deduction, abduction, induction) conditioned on a buffer of its own past problems and pushed toward variety, with a learnability reward tuning the difficulty. A code executor validates the proposed tasks and checks the answers, serving as one unified source of verifiable feedback. Trained on no external data at all, AZR reaches overall state of the art on coding and mathematical reasoning, past "zero" models built on tens of thousands of human-curated in-domain examples.
Sit with that architecture for a second, though. AZR escapes the human data bottleneck by building its curriculum inside a code executor, which is to say it solved the labeling problem by choosing the one domain where verification was already free. That isn't a criticism of the paper. It's the shape of the whole field, showing up again in the lecture that was supposed to be about the future.
Multiagent Finetuning takes on a failure that shows up whenever you run a self-improvement loop more than a few times: successive rounds hit diminishing returns, because the model keeps training on data that looks more and more like itself. The fix is to stop finetuning a model and start finetuning a society. Take a group of models from one base, let them interact, and specialize each on its own independent slice of the resulting data; generators and critics trained separately. Diversity holds up across rounds, and sometimes improves, where single-model iteration flattens it. This is the same thing DAPO watches for with its entropy metric, attacked at the population level instead of inside one training run.
Intelligence per Watt changes the subject, and belongs here for a reason every previous section earned: every technique in this course spends more compute at inference time, and someone pays that bill in joules. The Stanford and Together AI group (the author list runs through John Hennessy, Azalia Mirhoseini, and Christopher Ré) proposes IPW, task accuracy divided by unit of power, and measures it across 20+ local models, 8 accelerators, and 1 million real single-turn chat and reasoning queries. Local models answer 88.7% of them. IPW improved 5.3x from 2023 to 2025, with the share of queries a local machine can service climbing from 23.2% to 71.3%. Local accelerators still run at least 1.4x lower IPW than cloud accelerators on identical models, which is headroom rather than a verdict.
The instructors' closing list runs to 10 directions, and the ones that stayed with me are continual learning from test-time experience, infrastructure for high-throughput low-latency test-time scaling, generalization to non-verifiable domains, and finer-grained intelligence-per-watt measurement across tasks and hardware. Underneath all of them is the constraint the instructors state without softening it: when a model surpasses the experts you could hire to grade it, finding people who can set and score its work becomes the limiting step. DeepSeekMath-V2 is a bet that the verifier can be made to scale on its own. Nobody in this course claims to know yet whether that bet pays outside mathematics.
Last Takeaway
The number I keep returning to is 31%. That's the ceiling on DeepScholar-Bench, where no system tested clears a geometric mean of 31% at writing a related work section: a task the paper describes as hours of literature searching, reading, and writing for a human expert. In the same 9 lectures, AlphaCode 2 matches a million samples with 100 of them, and DeepSeekMath-V2 scores 118 out of 120 on Putnam. The distance between those two results is the course.
I've argued here that most of what reads as new capability is better selection, and that a selection gain travels exactly as far as your verifier does. What would change my mind is specific: a DeepSeekMath-V2-shaped result somewhere with no answer key at all. Train a faithful verifier for legal reasoning, or experimental design, or a codebase too large to unit-test, and show it holding its edge as the generator gets stronger. That's the experiment I'd read first.
If you're going to spend time on this material, start with the course site rather than the videos. Every lecture lists its paper readings with links, and the readings are where the arguments actually live; nearly every number in this post came out of a PDF rather than off a slide. Thanks to Azalia Mirhoseini and Aakanksha Chowdhery for teaching it, and to Stanford Online for publishing the lectures where someone like me could sit in the back and take notes.
If you're building agents in a domain where the verifier isn't free, and most domains aren't, I'd like to hear what you're doing about it. That's the part I still can't think my way through, and it's a better conversation than another benchmark chart.