Learn a shared representation contract.

A common representation is a contract

Text, images, graphs, board states, and robot observations can all be represented as numerical tensors. That does not make their meanings interchangeable. An embedding optimized to recognize “a cup” may omit exactly where the cup is. A controller may need position, orientation, units, and uncertainty as well as object identity.

The previous lesson used a programmed coordinate conversion. Here the adapter learns its four weights from paired examples. The sender is still a hand-defined teaching encoder, not a trained image or text model. This isolates one mechanism used in larger systems: aligning one component’s output with another component’s required input.

Learn a transformation from pairs

For each training location, the sender emits a vector and the receiver has a desired target. The complete sender converts [x, y] into [y, −x]. At first the adapter is an identity matrix: it passes the numbers through unchanged. The receiver therefore interprets the wrong coordinates.

The loss averages (predicted x − target x)² + (predicted y − target y)², divided by two, over eight training pairs. Backpropagation measures how each adapter weight affects this loss. Gradient descent updates those four weights at rate 0.2 while the encoder stays frozen. The ideal adapter’s rows are [0, −1] and [1, 0], which reconstruct [x, y].

Use the matrix refresher if multiplying rows by vectors feels unfamiliar. You can finish the main experiment by inspecting the plotted predictions and losses; deriving the matrix gradient is optional.

Optional: the adapter gradient

Let z be the sender’s two-number output, A the adapter, and t the desired position. Prediction is Az. For one pair, loss is half the sum of the two squared errors. The gradient for entry A[j,i] is (prediction[j] − target[j]) × z[i]. Average this over training pairs, then subtract 0.2 times the gradient. No gradient changes the frozen encoder in this lesson.

Missing coverage and missing information are different

With diagonal-only training, every example has x = y. Several transformations agree on that line. A low training loss cannot tell you which one will work elsewhere. The 16-position monitoring grid exposes off-diagonal mistakes. These positions never enter the updates, but repeatedly using them to choose settings makes them familiar development evidence rather than an untouched final test.

When the encoder discards y, the problem is more fundamental. Positions (0.3, −0.9) and (0.3, 0.9) both produce [0.3, 0]. A deterministic adapter must return the same answer for identical inputs. It cannot exactly reconstruct two different targets. More paired examples can reveal this ambiguity, but they cannot make the missing distinction reappear. A probabilistic decoder could express uncertainty over possible positions; it still would not know which unobserved y was present.

That is why useful systems may keep shared semantic representations, specialist-specific state, and typed exact records together. One universal vector is not a requirement.

Different objectives preserve different information

The lab uses a predictive objective: reconstruct the required coordinates. A downstream control loss could instead reward accurate actions or penalize position error. Its usefulness depends on which properties the task measures.

A contrastive objective teaches a shared space by comparing related and unrelated inputs. For example, a matching image and caption form a positive pair; other captions can provide negative comparisons. The objective favors the matching pair over alternatives. Merely pulling every pair together without a mechanism that preserves distinctions can produce a collapsed representation that says little about anything.

Contrastive semantic alignment can help retrieval, but a caption such as “a cup on a table” may not specify the precise position needed for grasping. Combining semantic alignment with prediction and task-specific objectives can preserve more useful information. No loss automatically guarantees that all future tasks’ requirements survive.

Typed structure is another shared language

A typed intermediate representation specifies fields and relationships that multiple components understand. Compilers use intermediate representations to connect stages without forcing every stage to work directly on the original source text. AI systems can similarly exchange entities, features, relations, legal actions, observations, and outcomes.

In the source’s chess example, a vision model first proposes a board state. A validator checks its structure and necessary state fields. An exact legal-move engine then applies the rules, a policy network scores legal candidates, and a language model explains the selected action. The explanation is not the authority for move legality. Necessary exact information includes piece identities and squares, side to move, and rule-relevant state such as castling rights and en-passant eligibility. A structurally valid but visually misread board can still be wrong, so perception uncertainty also matters.

For software, syntax trees and symbol graphs retain relationships that a vague description can omit. For robots, structured object states and control constraints preserve geometry and limits. The shared contract need not erase each specialist’s richer internal representation.

Graphs let relations guide computation

A graph neural network can update a node using its own state and messages from neighboring nodes. A common pattern is: transform neighbor states into messages, aggregate them with an operation such as a sum, then apply a learned update function. The graph’s edges decide which information can interact during a pass.

The lab’s three-node example uses a programmed binary propagation rule: keep a signal if this node or any neighbor had it in the previous pass. It is not a trained GNN. One synchronous pass moves information one edge; two passes can connect nodes two edges apart. The original typed edge directions remain explicit even though this demonstration deliberately allows messages both ways. Real message functions can use relation types and direction too.

Learned graph processing can score actions or infer useful relational features. It does not replace an exact rule checker when legal actions must be guaranteed.

Define a small game interface

The source exercise asks for three game entities and two typed relationships. Draft your own schema before revealing this example. Identify which fields must remain exact and which may carry estimates.

Example schema and exactness requirements

Entities: player p1 at cell (1, 2), key k1 carried by p1, and locked door d1 at cell (3, 2). Relations: p1 —carries→ k1 and k1 —unlocks→ d1. The carries relation accepts player→key; unlocks accepts key→door. Preserve exact IDs, types, cell coordinates, possession, lock state, and the environment version used to interpret an action. A policy score or estimated success probability can be approximate. An exact environment checks whether an open-door action is legal; a high policy score does not change the rules.