Learn AIBuild an agent.
Test the whole task.
Put learning, exact execution, search, memory, and evaluation together. Your final result is a reproducible experiment and an honest report—not a required victory over the strongest baseline.
For learning directly from rewards, try the separate reward-agent project. This puzzle project learns from an exact supervised teacher.
From predicting moves to finishing a puzzle
The blank can swap with a neighboring numbered tile. “Up” means the blank moves up; the numbered tile moves down. Reach the arrangement 1, 2, 3 / 4, 5, blank. A network can learn which moves appear useful, while an exact simulator checks the rules.
Before training, predict whether better single-move accuracy will always mean more solved puzzles. Keep that prediction in mind when the two measurements disagree.
Useful preparation: agent responsibilities, backpropagation, exact search, and evaluation design.
Choose with validation, then freeze
- Train 100 updates at a time. Compare training and validation move accuracy, policy loss, and value error. Keep up to two candidate checkpoints.
- Choose an expansion budget and compare complete-puzzle performance on validation. Try search, guided search, and visited-state memory. Change one factor at a time.
- Restore your chosen checkpoint and settings. Freeze once, then inspect both reserved standard positions and the held-out rule variant.
- Trace failures, write all five report sections, and record project completion. Export the report and back up editable work in My Work.
Loading the local agent workbench…
What this agent actually learns
The encoder is a fixed one-hot mapping: six positions × six tile identities gives 36 inputs. A 16-unit tanh layer feeds a four-action policy and a sigmoid value. The network has 677 learned parameters. The policy uses an exact legal-action mask; that mask enforces rules but cannot make the chosen legal move strategically good.
The teacher is an exact shortest-path solver under the standard rules. It supplies an equal target probability for each optimal move and a value target of 1 − distance / 21. The maximum standard-puzzle distance is 21 moves. Backpropagation minimizes policy cross-entropy plus weighted squared value error. Each update uses all 128 training initial states. Validation and reserved examples do not enter gradient updates.
What each controller contributes
The policy takes its most likely permitted action. Visited-state memory can prevent repeating known states but does not provide backtracking. Breadth-first search explores explicit alternatives and is an exact shortest-path baseline when its budget is sufficient. Guided search orders states by path length plus the network’s estimated remaining distance; that estimate need not be admissible, so optimality is not guaranteed. The adaptive controller is a programmed rule: try at most four policy decisions, then search with the remaining expansion allowance.
The node-expansion cap and 32-move cap bound work. A budget failure is not an impossibility proof. Model calls, legality checks, execution checks, training, and queue overhead have different costs. An equal expansion cap does not establish equal total compute or latency. Resource averages include failures; extra-move averages apply only to successes.
What the evaluation establishes
Training, validation, reserved-standard, and reserved-rule initial states are disjoint. A path may pass through a training position, and all standard cases share the same rules. The changed-rule set contains 12 solvable starts in which tile 1 may only move vertically. The runtime uses that rule’s mask and exact transitions; the policy/value weights remain frozen. This tests a specific rule shift, not transfer to all games.
Repeated feedback can overfit validation and reserved data. The project retains reserved exposure across weight resets, resumed development, and imported backups. After inspecting reserved results, obtain genuinely new cases for a new independent claim. A strong exact baseline or a failed architecture change is a valid scientific result.
The broader architecture lesson covers search-guided self-play, recurrent reasoning, learned compute allocation, and a multi-game research program. Those are separate experiments; this browser project does not claim to implement every research-scale component.
Check the claim you can make
Your agent chooses a shortest-path move on most validation positions but solves few whole puzzles. What should your report conclude?
Keep the evidence with the model
Your report should make the task, assumptions, choices, costs, failures, and limitations understandable to another learner. The completion record confirms a trained frozen experiment and filled report sections; it does not certify writing quality or model reliability.
Based on ebook lesson 34, pages 74–75.