Test the parts of a planning system.

Three operations, three roles

Recurrence refines an internal state. Search explores explicit alternatives. A world model predicts what actions will do. A system can combine them, but they are not interchangeable ways of obtaining intelligence.

For a deterministic puzzle with known rules, a practical design can use exact transitions while neural policy and value estimates guide which alternatives to explore. For unfamiliar physical dynamics, a learned model may support planning. A recurrent core might refine those estimates before the policy or value heads read them—but its contribution must be measured.

The world-model lesson isolates prediction error and corrective observations. This lesson holds transitions exact so that the comparison isolates refinement, its input representation, and its cost. Our refinement rule is programmed. We have not trained a recurrent neural core here.

Search can provide training targets

A policy network can propose move probabilities and a value network can estimate outcomes at search leaves. Search can improve on the initial proposal by exploring alternatives. Its improved action distribution and resulting outcomes can then become targets for later policy/value training.

That cycle connects search to learning, but it still requires meaningful search, reliable feedback, and evaluation of the updated network. A larger search does not automatically produce better targets, and targets generated from an inaccurate world model can inherit its mistakes. This lab compares inference procedures; running the comparison does not update network weights.

Test one component at a time

An ablation removes or changes a component to investigate its contribution. Our zero-pass setting removes refinement while keeping the combined search code. Search-only and zero-pass results should therefore agree. Discarding walls changes the refinement representation while leaving the exact transition simulator in search intact.

The combined method prioritizes frontier states using path cost so far plus a distance estimate. Unknown distance estimates contribute zero, so they do not pretend to supply a useful prediction. A partial refinement can be too costly for the help it provides. Counting its queries before search starts exposes that tradeoff.

The budget covers planning transition queries. Bookkeeping, sorting, and final verification are outside that counter. Equal query allowances are a controlled comparison of one resource, not proof of equal latency, energy, memory, or total computation. A deployment study should measure those separately.

A stronger evaluation than one solved map

The comparison includes two layouts absent from the earlier puzzle selector. These let you check whether the same procedure works under changed obstacles and starting positions. Repeatedly tuning against these fixed layouts makes them familiar; it does not create an untouched test set.

Record verified solve rate, moves, planning cost, and stopping reason. Distinguish a loop, an exhausted finite search, and a budget cutoff. Compare a simple direct rule, search only, and the combined method under the same conditions. A component can help some inputs and hurt others.

A controller could choose direct action for simple states and spend more work on uncertain ones. That decision also needs evaluation: uncertainty estimates can be wrong, and controller overhead costs resources. We return to learned routing in a later module.

Design the source’s Sokoban ablation

Sokoban involves pushing boxes to targets. A policy might learn to avoid pushing a box into a non-goal corner, while exact rules handle legal moves and search verifies a solution. A recurrent representation might try to refine estimates of trapped boxes. Those are proposed roles, not proof that every component is beneficial.

One controlled study design

Reserve puzzle layouts before training. Compare a direct policy, policy-guided exact search, and the same search with recurrent refinement. Keep observations, training data, action limits, and evaluation rules consistent. Count both refinement and search work within the allowance.

Separately remove recurrence, replace learned estimates with a simple baseline, and compare the proposed encoding with a version that preserves box and wall relationships. Report solve rate, invalid moves, deadlocks, and cost, including failures. Test selected combinations because components can interact. Use exact move replay and goal checks; do not infer a gain from a single attractive trajectory.