Learn AIPredict what an action will change.
Learn transitions from observations, compare imagined and actual routes, and test when replanning helps.
By the end: Explain action-conditioned prediction, model error, observation coverage, and the extra cost of observing and replanning.
Start with something concrete.
A map says a hallway is open, but you find a locked door. Continuing the original list of turns can take you somewhere very different from the imagined route. A world model is useful only when its predictions help actions work in the real environment.
Try the interactive model ↓A model correctly predicts most individual moves. Does that guarantee a long planned route is correct?
Change one thing. Watch what follows.
- Before collecting data, inspect square 1 with action Right on the first map. Compare the model’s optimistic prediction with the wall-constrained outcome.
- Run a trial with “Plan once” and 24 moves. Reveal the whole trial, then compare “Observe and replan”. Read the actual move and planning-query counts.
- Reduce the real-move limit to 8. Explain why replanning still does not guarantee success.
- Collect 100 transitions and inspect the same pair. Compare observed-pair coverage, one-step agreement, and trial results; do not call the coverage diagnostic a held-out test.
Loading this browser experiment…
From predicting a label to predicting a consequence
A classifier predicts a label from an input. A world model predicts how an environment changes after an action. The conditioning matters: the same state can lead to different next states depending on what the agent does.
In this deterministic grid, a state is a position and an action is a direction. Boundaries and walls determine the next position. Our small model learns a table of observed state–action outcomes. Before a pair is observed, it uses a movement prior that ignores walls. The table is learned from data, but the prior and learning rule are programmed; there is no neural encoder in this experiment.
Because the environment is deterministic, repeated observations of the same pair agree. A noisy environment would require a distribution over outcomes rather than treating one observation as certainty. An unobserved pair can be predicted correctly by the prior, so agreement alone is not proof that the model learned that transition.
Retain the state that the task needs
An image-based robot may use a neural encoder to transform observations into a compact state representation. A learned transition model then predicts the next latent state given an action. Predicting useful latent quantities can be cheaper than rendering every pixel of a future image.
Compression can also discard something essential. Omitting a small obstacle may make the predicted route look clear. The source’s sliding-on-ice example asks what position alone misses: velocity or motion direction, whether the robot is still sliding, friction or surface type, and relevant collisions can affect the next outcome. Two observations with the same position may require different predictions.
If the current observation does not expose a needed variable, a history or a state estimate may help. The requirement is not “store everything”; it is “retain enough information for the prediction and task.”
Plans can drift away from reality
A rollout repeatedly feeds a predicted state into the next prediction. An early model error can therefore change all later imagined states. High average one-step agreement does not guarantee a correct long route, especially when a few critical transitions are wrong.
The lab’s open-loop strategy computes a full action list once. If a move is blocked, later commands still come from the old plan. The trace’s imagined states belong to that rollout; actual states come from exact execution. This makes the divergence visible.
Model-predictive control plans ahead, executes the first action, observes the resulting state, and replans. Our trial additionally updates a temporary transition table from each observed move. Both strategies receive those observations, but only the replanning strategy uses them to choose a new route. Neither trial changes the saved model.
Replanning costs extra searches and still has a real-move limit. An inaccurate model, missing state, unreachable goal, or insufficient budget can defeat it. Collecting transition data beforehand can reduce those mistakes, but the same-grid coverage diagnostic is not a held-out generalization test.
Exact rules and learned dynamics have different jobs
When the rules are known, as in chess or a defined puzzle, an exact simulator avoids unnecessary prediction errors. Learned dynamics are useful when exact rules or relevant state are unavailable or expensive to model directly.
The source describes MuZero-style systems that learn representations and dynamics-related quantities useful for search, along with policy and value predictions, without reconstructing every visual detail. The lesson is about task-relevant prediction, not a claim that our transition table implements that architecture. Judge a world model by the decisions it supports and the errors it produces, not only by how plausible its imagined future looks.
A sliding robot’s next position depends on its current velocity. What is wrong with a model that uses position alone?
Take this idea with you.
A world model predicts consequences conditioned on state and action. Test its usefulness through executed plans, keep important state information, and budget for corrective observations.