Build a trustworthy practice loop.

A proposal is not a verdict

A generator proposes an answer or action sequence. A verifier checks specified properties. These jobs are different even when one system contains both components.

A Sudoku solution or legal game sequence can be checked against exact rules. Arithmetic can be checked by a separate exact calculation. Passing a collection of software unit tests is useful evidence, but tests can omit important cases; passing them is not proof of complete program correctness.

Our generator deliberately uses an imperfect carry threshold and sometimes flips its label. The exact checker recomputes whether a + b ≥ 10. The weak checker accepts any 0/1 label, including a wrong one. The table shows precisely what each check accepted.

Correct labels are necessary, but coverage matters too

Filtering incorrect proposals can produce trustworthy labels for the retained examples. It does not guarantee useful diversity. A generator that rarely solves a difficult class can leave that class underrepresented after filtering. Easy-only practice can also neglect the boundary where a model still makes mistakes.

A curriculum chooses which tasks to practice. It can move from easier to harder examples or adapt to current performance. Our adaptive rule chooses the first difficulty band below 85% validation accuracy, then samples training pairs from that band. When all bands pass, it practices the hardest band. This is a transparent programmed rule, not a learned curriculum controller.

The model’s inputs are the two digits divided by nine. Its sigmoid output estimates the carry label, with a 0.5 decision threshold. Training uses cross-entropy and actual gradient updates on accepted proposed labels. Initial weights are hand-set for the teaching example; they were not learned through a hidden pretraining run.

Three ways to use verified experience

Filtered supervised fine-tuning trains a model to imitate retained successful examples. The carry lab demonstrates this pattern with a small classifier, rather than fine-tuning a language model.

Reinforcement learning can learn from graded rewards over sampled trajectories. Feedback may concern a complete outcome rather than a desired action at every step. The earlier self-play lesson supplies an example.

Search distillation uses a stronger or more expensive search process to generate targets for a cheaper model. A searched solution can become training data after verification. A teacher’s limitations, biased coverage, and verification gaps can still enter the student. These approaches solve different feedback problems; none guarantees improvement simply by generating more data.

Promotion should protect what already works

The lab keeps the current model separate from a candidate. The candidate learns from a batch, then a gate checks validation results. It must improve hard-case loss by more than 0.001 and keep easy accuracy above the larger of 90% or the current accuracy minus five percentage points.

This explicit gate can reject a candidate that improves one measure while damaging another. Its thresholds are teaching choices, not universal quality standards. Other tasks may need checks for reliability, efficiency, calibration, or particular failure types. The final reserved pairs are excluded from training, curriculum selection, and the gate. Once their results guide further choices, reuse becomes practice rather than untouched testing.

Extend the idea to puzzles and code

The source’s Sokoban loop uses an exact simulator, solution verification, increasingly difficult layouts, training on successful searches, and transfer checks on reserved puzzles. Valid solutions alone do not prove that generated layouts are diverse or that a policy can solve unseen puzzles.

For the source exercise, define a promotion gate for a coding agent before viewing this example.

Example: protect earlier coding skills

Reserve new bug-fixing tasks and an earlier regression suite outside training. Require improvement on the new suite while keeping protected prior tasks above stated thresholds. Check that patches actually pass relevant tests, include adversarial or boundary cases, and respect a consistent tool/time budget. Inspect representative failures and compare multiple runs where behavior is stochastic.

Do not promote solely because the agent says its patch is correct. Passing these finite checks is evidence within their coverage, not a proof of correctness for all programs. Preserve the previous model so a later regression can be rolled back.