# Reward-agent implementation studio Python 3.10 or newer; standard library only. No account, package install, GPU or API key. This original public 3×3 teaching grid is a Project 10 implementation exercise. It prepares for P09; P09's documented standard simulator, geometry/control and richer scenario evaluation are still separate work. It is not a robotics environment. ## Start with your own implementation 1. Read the function contracts in `reward_exercise.py`. 2. Implement exact transitions, stopped targets and all online minibatch gradients. 3. Run `python3 reward_checks.py reward_exercise.py`. It intentionally fails before you implement the TODOs. The checks are exposed practice, not independent review. 4. Explain task termination versus a collection timeout, why target gradients stop, and why repeated samples still divide by the batch size. 5. Optionally compare with `python3 reward_checks.py reward_reference.py`. The supplied reference is a worked solution; running it does not make it your implementation. The finite-difference checks perturb online weights only and hold target weights fixed. They use every one of the 116 parameters, including zeros on untouched inputs. Transition checks use a separate edge table. Neither test establishes generalization. ## Train and resume the supplied reference Run each command from the extracted studio directory. Output files are created exclusively: choose a new name when a file already exists. ```sh python3 reward_reference.py train seed7-200.json --seed 7 --steps 200 python3 reward_reference.py resume seed7-200.json seed7-1000.json --steps 800 python3 reward_reference.py train seed19-1000.json --seed 19 --steps 1000 python3 reward_reference.py train seed41-1000.json --seed 41 --steps 1000 ``` The default uses replay, target copies every 20 updates, rate 0.01 and exploration 0.3. Alternatives: `--replay off`, `--target-every 1`, `--rate 0.05`, `--epsilon 0.1`. Apply alternatives to all three fresh runs when making a matched cohort. Resume retains its input settings and complete checkpoint, including target weights, 64-receipt replay, random state, episode position and counters. Maximum 2,000 total updates; numerical calls cap at 25. A failed run never replaces its input file. Train uses actual simultaneous SGD with a stopped gamma-0.9 target. The model has 9 one-hot inputs, 8 tanh units and 4 action values. One update collects one transition and fits eight samples. Newest-only mode repeats the newest receipt eight times, while consuming the same random draw budget. Ties use the lowest action index. Terminal goal/hazard receipts remove bootstrapping. A 24-move collection reset keeps the preceding receipt nonterminal. Counts are not measured equal-runtime claims. ## Freeze and inspect in the browser project Open `/learn/projects/reward-agent/` in LearnAI. Import the three 1,000-update JSON files, select them, choose move/search budgets, then freeze and explicitly open evaluation. The importer validates numeric state and recomputes outcomes; it cannot authenticate source code, seeds or training history. No reserved outcomes are included in the training export. The browser retains known evaluation exposure locally; copying files or using another device does not make previously seen cases unseen. Keep every seed and every failed start. Explain the full-model search baseline's information advantage and distinguish model calls, edge queries and execution moves. Locally reserved starts can have been visited by training trajectories. Public command-reversal cases are limited mechanism tests, not independent deployment data. ## Submit implementation evidence Attach your selected functions, checks and reproduction instructions as inert text in the project's implementation field. Keep complete source files locally. Compare your work with the reference only after attempting it, disclose that exposure, and explain one gradient check and one failed trial. Separate objective cases in the browser assess scalar/design transfer. Code authorship and quality still need human review; learner pilots remain pending. These public checks award no mastery. `manifest.json` records source sizes and SHA-256 hashes. `LICENSE.txt` covers the original code and authored synthetic task. There is no downloaded dataset.