Build the update.
Check the evidence.

1. Predict a target before writing its gradient

A target combines an observed reward with a discounted estimate of later rewards. Here the discount is 0.9: one point predicted for the next state contributes 0.9 now. A task terminal has no later reward. A timer stopping data collection is a different event; keep the actual nonterminal receipt.

Target = -1 + 0.9 × 3 = 1.70.

This is a freely chosen arithmetic probe, not a receipt from the project grid or a Python execution. While differentiating the current prediction's half-squared error, hold this target fixed. Change the terminal toggle and explain which path through the calculation disappears.

2. Download the exercise and public checks

Unzip into a local folder. Use Python 3.10 or newer; no packages, GPU, account or API key are needed. These commands execute local Python normally, outside the browser's restricted coding interpreter.

python3 reward_checks.py reward_exercise.py
python3 reward_checks.py reward_reference.py

The exercise intentionally raises NotImplementedError until you implement three functions. The supplied reference passes five public checks. Running the reference is guided execution, not evidence of your own implementation.

3. Implement, inspect and repair

1. Write exact transitions for the 3×3 grid. Actions are Up, Right, Down and Left; boundary bumps stay put. Goal 8 gives +10 and ends the task; hazard 5 gives −10 and ends it; other moves cost −1. Reject commands from terminal cells.
2. Compute the stopped target, suppressing the future term only for task termination. Explain why resetting a 24-move collection episode does not change its final receipt.
3. Implement the minibatch gradient of mean half-squared prediction error for a 9–8–4 network with tanh hidden units. Accumulate shared weights and divide by the number of receipts, including repeats. Use the same preceding parameters for every derivative.
4. Run the checks, inspect a discrepancy and add your own failing case. Explain the finite-difference step size and why only online parameters are perturbed.

The public checks use a separate transition table and numerical central differences for all 116 parameters. They inspect terminal targets, boundary bumps, repeated-receipt averaging and unchanged input models. These are exposed practice checks; hidden-case review and implementation authorship remain unverified.

4. Train and resume the supplied reference

python3 reward_reference.py train seed7-200.json --seed 7 --steps 200
python3 reward_reference.py resume seed7-200.json seed7-1000.json --steps 800
python3 reward_reference.py train seed19-1000.json --seed 19 --steps 1000
python3 reward_reference.py train seed41-1000.json --seed 41 --steps 1000

Default settings are replay enabled, target interval 20, rate 0.01 and exploration 0.3. Alternative flags are --replay off, --target-every 1, --rate 0.05 and --epsilon 0.1. Change one setting for a fresh matched cohort; resume retains the original settings.

Each update collects one transition and fits eight samples. Replay retains at most 64 receipts and samples with replacement. Newest-only mode repeats one receipt eight times while consuming the same random-draw budget. Counts are not equal-time measurements. The reference caps numerical calls at 25 updates and total training at 2,000.

A complete checkpoint stores online and target weights, replay, random state, episode position and counters. Resuming only the online weights would change the experiment. Output files are created only at new names; input files are not overwritten. Training exports do not open reserved evaluation.

5. Freeze, compare and explain

Import the three 1,000-update JSON files into the saved reward-agent project. Select the matched seeds, freeze move/search budgets, then explicitly open evaluation. The browser validates numbers and recomputes outcomes; it does not authenticate Python authorship or recorded training history. Keep every failed seed.

Compare learned, authored and full-model search policies. Explain the search baseline's information advantage and separate model calls, edge queries and executed moves. Locally reserved starts may have been visited during training. Repeated public command-reversal trials are practice, not independent deployment evidence.

Attach your selected functions, tests and reproduction instructions as inert text in the implementation report. Keep full source files locally and disclose reference exposure. Five separate scalar/design transfer cases are available, while human code review and learner pilots remain pending.

This tiny original grid prepares for the planned P09 simulator studio. It does not complete P09's documented standard-simulator integration, geometry/control work or richer scenario evaluation, and it does not provide physical-robot experience. Optional studio work does not change published completion requirements.