Decide whether another computation is worth it.

A router chooses; an orchestrator follows through

A router chooses a destination for a request. An orchestrator manages a process: the goal, subgoals, dependencies, working state, uncertainty, available resources, execution, verification, and stopping. It may calculate, search, retrieve, simulate, ask a specialist, gather evidence, or stop.

For a debugging task, “send this to the coding model” is one routing choice. “Inspect the failure, select a likely repair, run the relevant test, revise the hypothesis if it fails, and stop when the acceptance check passes or the budget is exhausted” is an orchestration process. New observations can change the next action.

The controller can be expressed as a policy: next computation = policy(goal, working state, uncertainty, remaining budget). This does not require the policy to be neural. Start with a clear rule that can be measured and inspected.

Put consequences and computation on a common scale

In this lesson, lower expected loss is better:

Expected loss = execution cost + waiting price × time + failure probability × failure loss.

This is one possible utility model. If success brings a separate benefit, subtract expected benefit from the loss, or maximize benefit minus costs. Never add milliseconds directly to money without choosing a conversion that represents the task’s priorities. Hard deadlines and execution budgets can also exclude an action regardless of its expected loss.

A fast, cheap model can be appropriate for a familiar task with low failure cost. A more expensive process may be justified when an error has larger consequences. Plausible wording or the model’s self-reported confidence is not an externally checked success probability.

The source debugging example, made explicit

The ebook compares a diagnostic costing 5 units with an immediate action carrying expected error cost 40. That comparison is incomplete until we also specify what the diagnostic reveals and what happens afterward.

Our default teaching scenario assumes a 40% chance of a dependency fault and a 60% chance of a local-code fault. Patching local code therefore fails with probability 0.4. With failure loss 100, its expected failure loss is 0.4 × 100 = 40. The patch itself costs 2, so the complete direct expected loss is 42, with waiting price initially zero.

The diagnostic costs 5 and is 90% accurate in each of the two fault categories. A dependency signal occurs with probability 0.4 × 0.9 + 0.6 × 0.1 = 0.42. After that signal, the dependency-fault probability becomes 0.36 ÷ 0.42 ≈ 85.7%. After a code signal, it is 0.04 ÷ 0.58 ≈ 6.9%. These are conditional probabilities, not the original diagnostic accuracy.

The plan repairs the dependency after the first signal and local code after the second. Across both branches it chooses the wrong repair 10% of the time. The diagnostic plan therefore has expected loss 5 + 2 + 0.1 × 100 = 17. Its net value relative to direct action is 42 − 17 = 25. The widget recomputes the branches when assumptions change; the number 25 is not a universal value for diagnostics.

Why averaging both branches matters

The controller does not know the signal before paying for it. It must weight each possible result by that result’s probability, choose the best eligible action after that result, and average those conditional costs. Selecting only the favorable branch would make the diagnostic look better than it is. A zero-probability signal has no defined posterior in this model and cannot be treated as an observed outcome.

Information is useful when it improves a decision

Value of information asks how much better a decision can become after acquiring evidence. Net value subtracts the acquisition cost. A 50%-accurate symmetric diagnostic in this two-cause model provides no information. If it costs something, it cannot improve the optimal decision here.

Even informative evidence can be poor value. The likely cause might already be obvious, the remaining actions might all be the same, failure might be inexpensive, or obtaining evidence might consume the resources needed to act. In the lab, the diagnostic takes two time units. A deadline of two leaves no time for a repair afterward. “The diagnostic fits” and “the complete diagnostic plan fits” are different claims.

The 18-unit rebuild gives another alternative when the budget permits it. It has an assumed 2% failure probability, not perfect reliability. Compare the diagnostic with the best eligible direct choice, not a conveniently weak baseline. All these probabilities and costs are invented assumptions for a finite decision model; they do not estimate actual debugging performance.

Deciding how much to think

Metareasoning includes deciding whether another search expansion, refinement step, retrieval, or verification is worthwhile. Continue when the expected improvement in the resulting decision exceeds the added cost and fits the constraints. Stop when the task is verified, remaining work has poor expected value, or the budget is exhausted. Report unresolved uncertainty instead of treating a budget stop as success.

The lab allows at most one diagnostic and one repair. Real workflows can use longer histories, but each additional step must account for the updated state and resources. Repeating a diagnostic does not automatically provide independent evidence: correlated measurements can repeat the same error.

How a controller can learn

Supervised demonstrations can teach common choices. A contextual bandit can learn cost-sensitive selection from task features, chosen actions, and outcomes. Reinforcement learning can optimize longer sequences when the environment provides trustworthy rewards and controlled exploration. These approaches solve different parts of the problem.

Keep execution and verification outside the controller’s unsupported claims. A chosen tool call must still pass the runtime’s checks. Compare a learned controller with a simple rule using the same tools, tasks, and budgets. Added parameters alone do not demonstrate better orchestration.

Design a utility for a new task

For the source exercise, choose an agent task and specify success, execution cost, waiting cost, failure severity, and any hard constraints. Explain where each probability estimate would come from.

Worked transfer: checking a practice calculation

Suppose a direct answer costs 1 and is correct on 90% of comparable held-out practice cases. An exact calculator costs 4 and, under the assumed valid-input contract, succeeds on all of them. With failure loss 10 and no waiting cost, direct expected loss is 1 + 0.1 × 10 = 2, below the calculator’s 4. With failure loss 100, it is 11, so the calculator is preferred if it fits the budget. A malformed call can still fail; the probability estimate must include the actual runtime behavior. These numbers specify the exercise, not a measured product recommendation.