Help specialists understand each other.

A good specialist can still be a bad teammate

A vision component might locate objects accurately while a controller moves accurately to numerical coordinates. Connecting them can still fail if one uses centimetres and the other expects metres, or if they disagree about the coordinate frame. The interface is part of the system’s behavior.

The cup lab uses exact programmed observations so you can isolate this boundary. It does not run image recognition, learn a latent representation, or control hardware. Its position tolerance checks one property of a proposed point; a real grasp also needs obstacle avoidance, timing, force limits, and uncertainty handling.

Three ways to organize computation

Architecture	What is shared	Main tradeoff
Unified model	Many capabilities use shared parameters or representations	Shared learning may transfer useful structure, but updates can interfere
Modular system	Independent specialists connect through explicit interfaces	Components can be updated separately, but coordination and compatibility need testing
Hybrid	Some representations or processing are shared, while other components specialize	Flexible boundaries add design choices and evaluation work

None is automatically best. A single model may handle several tasks well. Separate vision, speech, planning, and control systems may need different data or computational operations. Compare complete systems under relevant constraints rather than counting their modules.

Choose what crosses the boundary

Text is flexible and readable. The sentence “the cup is on the right” communicates a broad location but loses precise coordinates. That is a limitation of this particular summary, not proof that language cannot express numbers: “x = 60 cm, y = 30 cm in the table frame” preserves much more. Inspect what the actual message retains.

Structured records make fields, units, and constraints explicit. The lab sends an object type, coordinate frame, unit, x, and y. The receiver rejects a missing unit instead of guessing. A structured record can still be incorrect; validation needs both schema checks and tests against the world.

Continuous vectors can represent rich learned information. Their meanings depend on the transformations that produce and consume them. Two independently trained models need not agree about coordinate meanings even if both use vectors of the same length. Our hand-defined example makes this visible: [y in cm, x in cm] cannot be passed straight to a consumer expecting [x in m, y in m].

An adapter transforms one representation into another. The lab’s adapter swaps and scales known coordinates. In more complex systems an adapter may be learned from paired examples; learning it and evaluating generalization require a separate experiment. A conversion cannot recover information the sender has already discarded without relying on additional assumptions or evidence.

Test meaning, not just shape

Scaling the lab’s vector without swapping it produces an in-bounds but wrong target at asymmetric positions. At x = y, the bug disappears. This illustrates why one easy successful example is weak evidence for compatibility. Test asymmetric coordinates, boundary cases, missing metadata, and changes in the upstream component.

For a chessboard, prose may describe a threat, a structured board can preserve every piece and whose turn it is, and a latent vector may help a learned evaluator. An exact move checker needs a board representation with the necessary rules and history; a vague summary alone cannot establish move legality. Choose the representation according to the downstream question.

Graphs preserve relations

The source’s video example combines visual event detection and speech transcription, potentially computed in parallel. An event graph can retain event nodes, timestamps, sources, and relations such as “before,” “after,” or “refers to.” Flattening everything into one sentence may hide which component observed what.

The lab’s two supplied events show a cup placement at 2 seconds and speech at 3 seconds. An audio clock offset changes the apparent order. Correcting known offsets aligns the timeline; it does not prove the speech is truthful or that one event caused another. Real event fusion also needs uncertainty, identity matching, and handling of missing or contradictory evidence.

Internal experts are a different boundary

An internal mixture-of-experts layer routes activations through selected expert blocks inside a network, often for individual tokens. A modular system selects independent full models or tools for subtasks. They both use specialization, but their interfaces, training, and decision granularity differ. A modular system can itself contain a language model with MoE layers. Review the distinction between total and active weights in compute budgets.

Design a cup-picking interface

For the source exercise, assign information to each responsibility: language can state “pick up the red cup”; a structured world state can store object identity, pose, frame, units, uncertainty, and obstacles; continuous control values can specify bounded joint targets or velocities at the required rate. Preserve timestamps across the handoffs. A controller should stop or seek better evidence when a necessary constraint is missing.

The goal is not to force every specialist into one format. It is to preserve the distinctions that the next computation needs and verify that the components interpret them compatibly.