Adapt a pretrained model.

Start from useful learned parameters

The preceding decoder learned a tiny grammar through actual gradient updates. We now reuse its 1,000-update checkpoint as a starting point. That earlier learning is its pretraining relative to this new task. It is an original teaching model with a tiny vocabulary, not a general-purpose language model.

The new grammar swaps red and blue at the final color, but leaves green and gold unchanged. “Red cat sees red” becomes “red cat sees blue”. Both adaptation candidates start from exactly the same source weights and receive the same twelve training combinations. The four development combinations stay outside these updates; they remain public, repeatedly inspected examples from the same generator.

Separate four ways to change behavior

Pretraining learns parameters from an earlier training objective. Prompting supplies different input context while keeping parameters fixed. A prompt only helps if the model can use the information provided; this tiny decoder has no representation for an arbitrary English instruction such as “follow the new rule”.

Instruction tuning is supervised training on examples that pair instructions or requests with desired responses. It changes parameters. If two tasks need different answers to otherwise identical inputs, the instruction or another task cue should distinguish them in the data. Low-rank adaptation describes how a parameter update is represented, not what supervision it uses: an instruction-tuning objective could use either full parameter updates or low-rank updates.

Our lab performs supervised adaptation to a changed grammar. It does not secretly add an instruction token, train a natural-language instruction follower or optimize preferences. The distinction matters: a compact update cannot repair an underspecified task contract by itself.

Compare full changes with a constrained update

Full fine-tuning changes the decoder’s active embeddings, attention matrices, feed-forward parameters and output parameters. Of its 254 stored values, 250 participate in training; four end-token embedding values are unused and stay fixed.

The alternative freezes the entire base model and adds a learned update only to its output matrix. Its four input features map to ten output logits. Instead of learning an arbitrary 10 × 4 update directly, we multiply two smaller matrices: A maps four features into two intermediate values, and B maps those two values into ten output changes.

Here W_base is the original 10 × 4 matrix. A has shape 2 × 4, B has shape 10 × 2, and BA has shape 10 × 4. The effective matrix adds that update to the original. The update has rank at most two: its changes are built from only two intermediate directions. This constraint can be useful without being sufficient for every task.

This is a deliberately small, output-only version of the frozen-base, low-rank-update approach described in the LoRA paper. The lab’s scale factor is one. It does not reproduce the paper’s model sizes, target layers or performance results.

Work through one matrix entry

To update one output coefficient, multiply the two corresponding pairs and add them. If a selected B row is [0.2, −0.1] and the matching A column is [0.3, 0.4], the update is:

The symbol Δw means the change in that coefficient. An original coefficient of 0.50 becomes 0.52. Use the live row and column selectors to reproduce the same calculation with the current learned matrices. The output bias stays frozen in this adapter experiment.

A contains eight trainable values and B contains twenty, for twenty-eight total. The base weights are still needed for inference. A smaller adaptation file does not mean the whole model has only twenty-eight parameters, nor does parameter count alone establish memory use or speed.

Initialize without changing the starting prediction

B starts at zero while A contains small nonzero values. Their product is initially zero, so enabling the adapter initially leaves the base prediction unchanged. B receives a gradient through the nonzero A values; after B moves, A can receive a gradient too. Starting both matrices at zero would give both zero gradients for this bilinear update and stall learning.

Each training step computes the mean next-token loss over all sixty training targets, including end. Full fine-tuning differentiates through the whole decoder. The low-rank branch reuses frozen input features and differentiates through A and B. Both use simultaneous pre-update gradients and their own global gradient clipping at norm one. The same rate and number of updates do not produce identical changes in these different parameter spaces.

The source checkpoint remains separate and unchanged. Disabling the adapter restores its original weights exactly; enabling the adapter reapplies the learned change. That gives a way to select which model to use, not a model that inferred an unprovided instruction.

Evaluate new behavior and retained behavior separately

The original and new evaluations use the same four prompts. Two of them demand different final colors under the two protocols. A deterministic response cannot satisfy both labels at once. If adaptation learns the new rule perfectly, the original final-color score necessarily loses those two conflicting cases. Do not describe that result alone as evidence of unrelated catastrophic forgetting.

The other token positions provide additional checks: did prediction of the first color, animal, sees or end change? Compare their original-task losses as well as the final-color accuracy. A method can learn the requested rule while creating other regressions, and a lower mean loss can hide a poor answer at one important position.

For a system that must support both rules, supply an explicit rule cue and train/evaluate the conditional behavior, or deliberately select the appropriate model externally. This lab demonstrates the second option by enabling or disabling the adapter. A more realistic instruction-tuning task requires its own dataset, evaluation and retained-capability checks.

Make a defensible adaptation decision

For guided practice, explain one matrix coefficient and compare predictions with the adapter on and off. For implementation practice, verify every adapter gradient with a small positive/negative perturbation, ensure the base never changes, and reproduce a restored update. For a decision brief, name the required behavior, the conflicting versus unrelated regression risks, the evidence supporting a candidate and the additional real-text checks needed.

Capture matched comparisons before changing settings. Checkpoints and notes in this lab are temporary; export them before leaving. Pretraining and adaptation counts remain separate records, not authenticated training history. Reports, synthetic successes and unit tests do not substitute for independent implementation review or learner pilots.