Published objectives set $c\equiv0$. Fine where the checker is free — math, code. Not fine where an instrument is involved.

Most such comparisons are (i) on a small number of model families, (ii) at a fixed compute budget that favours one method, and (iii) evaluated with pass@1, which is exactly the metric Part IV shows can move without capability moving. Treat "RL generalizes, SFT memorizes" as a well-supported tendency, not a theorem.

Reward models are trained on comparisons drawn from a narrow policy. Once RL moves the policy, the model is evaluated off its training distribution — and its accuracy there is not measured by the validation loss that was used to select it. Ranking accuracy of most preference-tuned models is below 60 %, and correlates with win rate only near $\piref$.

PPO-versus-DPO results are extremely protocol-sensitive: reference model, data source, epoch count, and whether the preference data was generated by the same policy being trained. A study that reuses public offline pairs is testing a different question from one that regenerates pairs on-policy.

"Once a model-based PRM is introduced, it inevitably leads to reward hacking."

A veto constraint is a confession: linear scalarization of criteria is hackable, so the fix is a hard filter rather than a better weight. Expect the same pattern anywhere a learned judge sits inside the loop.

No monotonicity guarantee; behaviour depends on $\varepsilon$, on how many inner epochs are run per rollout ($\mu$), and on whether the sampler matches the trainer. All three are engineering choices that the theory does not cover.

Template, not RL. Qwen2.5-Math-7B average: R1 template 0.0, Qwen template 26.5, no template 38.2. A prompt-format choice moves the score more than most RL ablations.

"Aha" is not emergent. Self-reflection keywords are present in base models, including DeepSeek-V3-Base, before any RL — measured over 40,000 responses; nearly half the self-reflecting responses are no more accurate than the rest.

Model family, alignment state, reward scale, response-length cap and data difficulty all flip the sign of individual tricks. The DAPO ladder is a Qwen2.5-32B-base result; the Lite PPO result is Qwen3-4B/8B. Neither generalizes by default.

$s_i=\exp(\overline{\log w})$ is a geometric mean of per-token ratios, not the sequence importance weight $\prod_t w_t$. GSPO is a deliberately biased surrogate chosen for bounded variance — the paper argues from variance, not from unbiasedness.

MiniMax reports clip-higher was less effective in their setup — 16 off-policy update rounds per generation batch. The number of inner epochs $\mu$ silently changes which trick works.

No ablation isolates the $\ell_2$ proximal term against clipping. The claim "mirror descent beats PPO-clip" is not established by this paper — it is an alternative with a clean derivation.

The sequence weight is a product of $T$ token ratios. If each token contributes a small $\chi^2$ divergence, the variance of the product grows like $(1+\chi^2)^T-1$: exponential in length. Clipping each factor bounds the factors, not the product.

Any claim of the form "method X beats Y" from a single budget is a claim about one point on two curves. Ask for the curve, or at least for two budgets.

An execution reward is a sound verifier for "the tests pass" and an unsound verifier for "the bug is fixed". RL optimizes the first. The gap is exactly the coverage of the suite — which is why held-out tests, unavailable to the agent, are not optional.

In agentic RL the inference fleet is larger than the training fleet. This is not an implementation detail: it means the cost driver is environment interaction, and the research lever is sample efficiency, not kernel efficiency.

Reported settings: retained reasoning across tool calls and compaction instead of truncation. 2.9× in the reported public-set comparison. self-reported

"A 50% time horizon of X hours does not mean we can delegate tasks under X hours to AIs."

Absolute Zero and R-Zero report transfer evaluations on held-out benchmarks. Those finite results do not establish open-ended scientific capability growth; inspect task independence, coverage and evaluation budgets.

"Reward is enough" is a hypothesis, and it has a published objection: scientific objectives are irreducibly multi-objective — novel, correct, useful, cheap — and scalarizing them is a modelling choice, not a fact about science. Both positions are live; the lecture does not settle it.

Tests encode a spec, a simulator a model, an assay a hypothesis. The translation is where the error lives.

"AI solved the IMO" spans all four rows. Naming the tier is the difference between a machine-checked artefact and a press release — the practice Part VI asks for.

Nothing published has reached T-D.

If the reward is a proxy, measure the rank correlation between the proxy and the thing you care about before optimizing it, and report it. This is a one-day experiment that would have prevented years of argument — and it applies verbatim to every learned reward model in Part II.

MLE-bench has no controlled human-time baseline — the thresholds are Kaggle medals, and time budgets differ across leaderboard rows. It measures relative agent progress, not "agents versus scientists".

Despite a large GFlowNet literature on molecule generation, no primary source could be found reporting GFlowNet-designed molecules that were synthesized and assayed. The formalism is elegant; the wet-lab evidence is missing. That is an opportunity, stated as such.

Recomputed old scores are not necessarily sampling-distribution scores. Audit temperature, mode and stochastic forwards.

p4-grpo-v1: reward up, evaluation flat; logged entropy field halved — not proof of entropy collapse. p4-final: nonzero clipping, despite one inner epoch.

Asymmetric bounds: v1 0.2 / 0.28; v2 and final 0.2 / 0.30. p4-final logged nonzero clipping. One inner epoch does not guarantee ratio one: audit scoring modes and stochastic forwards.

Steps 12, 22, 30, 31, 39, 42, 44, 69, 79: every group in the batch was unanimous, dynamic sampling skipped all of them, and the logger recorded a sentinel 0 for entropy, KL and loss — the same mechanism that killed two runs entirely, appearing intermittently here. Illustrative homogeneous-Bernoulli arithmetic at the gate's $p=0.19$ and this run's $G=6$ gives ≈29 % unanimous prompts.

Two agentic-SFT runs with byte-identical recorded configurations differ 0.0 versus 1.0 in tool-call validity at essentially identical harness fit (token accuracy 0.44199 vs 0.44207). The delta is measured; the cause is not recoverable from the artifacts, because the run manifests do not persist the initialization lineage. Reproducibility is a data-model problem before it is a science problem.

Numerical conformance is enforced and passing. Task-quality retention is specified but not delivered: the suite pins ten benchmarks at a 0.95 threshold, and the results database currently holds one complete suite, one partial, and eight empty. The only end-to-end quality artefact is an 8-case smoke test on which both engines scored 0.125.