Build an architecture that earns its complexity.

Start where correctness can be checked

A tractable domain makes a research question testable. Card games, board games, and puzzles supply explicit rules, structured states, legal actions, objectives, and termination conditions. A typed task interface should expose those fields consistently while preserving each game’s actual semantics.

The ebook proposes games with different mechanics—FreeCell, Sokoban, chess, and 2048—as a research program. Their differences matter: hidden information, irreversible pushes, adversarial responses, and random tile arrivals demand different assumptions. A shared vector length alone does not make their interfaces compatible.

Our required browser project starts with one small, fully observable 2 × 3 sliding puzzle. It has 360 positions reachable from the goal under standard rules. A second rule forbids horizontal movement of tile 1, leaving 120 goal-reachable positions. The project’s 12 reserved rule-variant starts are distinct from all supplied standard training, validation, and test starts. It is deliberately smaller than the multi-game research program.

Five components to build and measure

A state encoder turns a valid observation into usable numerical features. Our fixed one-hot encoder gives 36 inputs. A policy/value model proposes moves and estimates remaining difficulty; our 16-unit tanh network has 677 learned parameters. An optional recurrent reasoning core can refine an internal state, but should be added only when a nonrecurrent baseline exposes a need.

A small meta-controller decides whether to use the policy, search, or another computation under a budget. Our adaptive controller is programmed, not learned: at most four policy decisions, then exact search. An offline learning coordinator collects suitable targets, updates parameters, compares candidates, and keeps evaluation separate. The project trains from exact standard-rule solutions and freezes before reserved evaluation.

Keep simulators, legal-action masks, search procedures, and evaluation gates as conventional software initially. These components establish enforceable contracts and reliable feedback. They do not have to be neural to be useful.

The ebook suggests 256-dimensional representations and tens of millions of parameters as an experimental scale worth investigating. Those figures are not a universal optimum or a prerequisite for learning the architecture. Our browser experiment uses 677 parameters so the computation and its limitations remain inspectable.

A disciplined training ladder

First train an encoder and policy/value head using known solutions or search targets. Confirm that gradients are correct and check held-out prediction error. Then investigate search-guided self-play: use search and verified outcomes to generate stronger targets, while controlling distribution changes and evaluating independently.

Compare fixed and adaptive search budgets under explicit resource accounting. Only then consider recurrent latent refinement, against strong nonrecurrent baselines and with its extra computation included. Finally, train a meta-controller from records of which strategies succeed efficiently, using controlled exploration and separate evaluation.

The browser project implements the first stage and compares policy, exact search, learned-value guided search, and a programmed adaptive controller. It does not perform search-guided self-play, train a recurrent core, or train its compute-allocation rule. The earlier self-play, recurrent refinement, and neural router lessons explain those separate mechanisms and their limits.

A better local score is only the beginning

The initial puzzle network can learn to choose optimal moves on nearly all training starts while doing little better on validation starts. That is evidence of fitting a finite training set, not of a general puzzle-solving algorithm. Complete attempts can still loop or exhaust their budgets. Evaluate whole-task success alongside individual predictions.

Learned-value guided search uses a fallible distance estimate. Its priority is path length so far plus predicted distance remaining. It may expand states in a useful order, or add overhead without improving outcomes. Since the estimate can overstate true distance, do not claim an optimality guarantee. Exact breadth-first search remains a strong baseline for this small state space.

Stored experiences or visited states can reduce repeated errors. They also introduce assumptions about identity, task versions, capacity, and reuse. Test their contribution with an ablation; do not infer usefulness merely because the architecture contains memory.

What a convincing research result would show

Measure transfer, not just training fit. Does a shared representation adapt to new mechanics with fewer examples? Does adaptive computation improve success for the same total resource cost? Does memory reduce repeated errors? Does verified new training improve unfamiliar tasks without forgetting older ones?

The source worked example trains on selected grid puzzles and card games, then holds out rule variants. Compare the shared agent with individual specialist models and exact search under a matched workload and resource protocol. Specify the examples, tools, action contracts, adaptation allowance, and training costs each system receives. Keep a genuinely unfamiliar game or rule combination reserved until the design is frozen.

The project tests one concrete held-out rule, with no weight adaptation after freezing. Its execution layer knows the new legality rule. This isolates one kind of transfer; it does not mean the model discovered the rule from language or learned a new game unaided. Initial states are disjoint, but solution trajectories can overlap training positions. Report that limitation.

Design a twelve-week experiment

For the source exercise, write a plan containing a baseline, two architecture changes, and a reserved evaluation set. The workbench timeline lets you choose two changes and inspect a staged schedule: establish the baseline, test each change, check their interaction, freeze, evaluate, then report.

An example is a policy/value baseline followed by learned-value guided search and visited-state memory. Before evaluation, define a success criterion such as more solved unfamiliar starts without exceeding a declared compute budget or regressing on a critical subgroup. Also record negative results. A second change that adds cost without benefit should not survive just because it makes the architecture look more complete.

Twelve weeks is the exercise’s research-planning horizon, not a claimed duration for completing this course. The required project runs locally and produces a small evaluated agent plus an exportable report. The broader architecture remains a hypothesis whose components must earn their place through evidence.

For an optional next step, open the research notebook to inspect two exact game contracts, compare non-neural baselines, and draft a frozen evaluation protocol. Its twelve-week guide distinguishes the working starter from later experiments you can implement.