Use a pretrained model responsibly.

Start with a concrete contract

A school application must answer a closing-time question from an approved notice. Before choosing a model, write down the input, permitted source, expected output and fallback. Here the weekday answer is 17:00; holiday hours are unknown. An attractive sentence or a correctly shaped object can still violate that task.

Entering here directly: a model consumes numbers called token IDs and scores possible next tokens. A tokenizer maps text to those IDs. An output schema describes the shape another program expects. Evidence connects a particular claim to a relevant source. None of these requires prior programming to inspect in the lab.

Text and token IDs must agree

“17:00” is five characters but three tokens in our teaching tokenizer: “17”, “:” and “00”. Other tokenizers split it differently. The teaching tokenizer lowercases text, splits words, digits and punctuation, and maps unfamiliar pieces to ID 0. Its unknown token loses information: different unfamiliar words become indistinguishable to this model.

GPT-2 uses a different tokenizer and vocabulary. In its recorded trace, inspect the space at the beginning of some pieces. A token ID has meaning only under its matching vocabulary. Changing the tokenizer while retaining model weights does not automatically teach the model the new IDs. The lab rejects this mismatch before inference.

Count every part of the budget

A context window bounds how many tokens the model can process in one sequence. In this exercise, reserve room for the continuation before running: input tokens plus reserved output tokens must not exceed the limit. For example, 60 input tokens plus 4 output tokens use all 64 teaching slots. Reserving 8 instead would require 68 and must be rejected or explicitly shortened.

In a chat application, count the serialized instructions, history, tool results and format overhead under that model’s actual tokenizer and template. Some services also impose separate output limits. A character estimate is useful for rough planning, not exact admission. Removing source material to fit may remove the evidence needed for an answer; review what was removed.

What the live model actually learns

The teaching model counts adjacent token pairs in five visible fictional sentences. It is a bigram model: its next-token prediction depends on the last token only. Its 64-token limit is an application teaching contract, not evidence of a 64-token reasoning ability.

After “closes”, the training corpus contains two transitions to “at”. To avoid giving every unseen transition zero probability, add one to each possible next-token count, then divide by the sum of these adjusted counts. With a vocabulary of V tokens, “at” receives 3 divided by (2 + V). V means the number of allowed token IDs, including the unknown token. The live table shows that arithmetic and all probabilities on a shared zero-to-one scale.

Greedy decoding chooses the largest probability, appends its token and repeats. This can produce loops. It does not consult the school notice, verify truth or learn from your prompt. A higher next-token probability means a stronger continuation preference under this model, not a calibrated chance that a claim is true.

Inspect an actual pretrained model

The second panel contains genuine CPU recordings from the 124-million-parameter GPT-2 model, with a pinned revision, asset hashes, tokenizer IDs and generation settings. GPT-2 is a historical text-completion model, not an instruction-tuned assistant. It is larger than the teaching model but does not represent current frontier performance. The lesson needs no download: all three traces are supplied locally.

The weekday run starts with a correct answer, then invents another question. The JSON-request run continues with unrelated fields and stops mid-object. The holiday run supplies an unsupported schedule. Inspect all three, including the 24-token stopping limit. These selected task types demonstrate possible contract failures; three teaching cases cannot estimate their frequency in general use.

Optional engineering depth: reproduce the traces

The repository recorder at scripts/record-pretrained-traces.py uses local safe-format weights only, checks their hashes and pins its model packages. It makes no model download automatically. The exported trace records exact inputs, outputs, the top three next-token probabilities and measured CPU time. Replaying elsewhere can introduce small floating-point differences; compare token sequences and probabilities with declared tolerances. Required browser learning does not execute the large model.

Separate syntax, schema and support

First, can the response be parsed as one complete JSON value? Extra prose, Markdown fences or a token-limited unfinished object fail this check. Next, does it contain exactly the required fields with the allowed types? Finally, do those values answer this question using the permitted evidence?

Our proposed 18:00 answer passes parsing and schema checks but fails the weekday source check. Repair it to 17:00. Then switch to the holiday question: the same answer now lacks relevant evidence. The agreed fallback uses null for both fields. Null means “no supplied value”, not zero or midnight.

Practice: a correct answer in the wrong shape

A plain sentence stating 17:00 may answer the weekday question correctly but still fail a downstream program that requires JSON. Preserve both tests: factual correctness alone does not make it machine-readable, and machine-readability alone does not make it correct.

The exercise uses exact comparisons against authored source facts. A general application needs a broader evidence policy and evaluation suite. Even schema-constrained generation, when available, does not itself establish relevance, authorization or truth.

Apply this to your path

Guided experiment: export a comparison with one format failure and one source-support failure. Explain which check catches each and why the model probability does not resolve either.

Implementation: write a wrapper that rejects mismatched tokenizer versions and over-budget inputs, validates an exact schema and routes unsupported answers to a fallback. Test unknown words, empty input, exact budget boundaries, extra fields and missing source facts. Compare your wrapper with the public lab cases; passing them is practice, not independent certification.

Decision brief: decide whether to use the model for the school task. Cite the observed failures, specify a fallback and propose new cases needed before deployment. Do not infer a production success rate from these three traces.

Continue to answer with evidence for claim-level checks or measure the whole workflow to examine task success and costs. All practice is optional to exploration; no earlier completion locks these routes.