Build a reproducible language-model experiment.

Choose the evidence you want to produce

Use the three linked browser labs. Capture an early and late decoder, compare temperature on the same weights, then inspect adaptation and retained behavior. Export your work. You can read the frozen example below without installing software.

These are optional entry points, not separate ability levels or new completion requirements. You can change depth freely.

Try: read a reproducible checkpoint

Recorded supplied run: seed 1, learning rate 0.3; 254 stored parameters, 250 active. These metrics come from the checked-in reproducible checkpoint artifact, not a new training run or a held-out learner assessment.

Training loss

0.5072 nats/target

Development loss

1.2983 nats/target

Development copy accuracy

4/4

Predict why copy accuracy can improve while development loss eventually rises. Open the position-level loss table in the trainable decoder experiment to check the missing animal combinations. One mean score does not describe every behavior.

Experiment: train, freeze, replay, adapt

Download CPU language studio

Extract the archive. Use Node.js 22.18 or later with built-in TypeScript stripping. No npm installation, GPU, server or network service is needed. The archive includes thirteen files and a byte/hash manifest. Run these commands from the extracted directory, with new output filenames:

node --experimental-strip-types studio.mjs train --steps 1000 --rate 0.3 --seed 1 --out trained.json
node --experimental-strip-types studio.mjs freeze --input trained.json --out frozen.json
node --experimental-strip-types studio.mjs evaluate --input frozen.json --out evaluation.json
node --experimental-strip-types studio.mjs adapt --input frozen.json --steps 100 --rate 0.3 --out adaptation.json

1. Train: actual full-batch causal decoder updates, capped at 1,000. Each engine call performs at most 25 updates. Output contains weights, seed, rate, runtime and source/corpus hashes.
2. Freeze: fix weights and a three-temperature sampling plan before computing the report. Freeze does not make an already public development set unseen.
3. Evaluate: recompute training/development losses, position-level failures, a training-only count baseline and sixteen samples per temperature. Same parameters and seed produce the same outputs.
4. Adapt: compare full updates with a rank-two output adapter from the frozen checkpoint. Report the changed rule and original behavior separately.

Files are created exclusively; existing outputs are never overwritten. Inputs are bounded to 500,000 bytes and JSON is never executed. Ctrl-C stops the foreground process; no final artifact is written until that command finishes. There is no partial-checkpoint recovery: rerun with a new output filename. The CLI and browser have different interruption mechanisms.

Understand: separate supplied execution from your implementation

The four commands above run a supplied working engine. They do not call your tokenizer exercise. To implement the first pipeline stage, edit learner.mjs, following its explicit token and document-boundary contract, then run:

node --experimental-strip-types studio.mjs check

This command deliberately fails until you implement the TODO. Return five target IDs with end, omit start, reject unknown words and malformed or mismatched-color sentences. Add your own counterexamples. Passing the small public check is practice, not independent grading.

Next reproduce a forward pass, the training objective and simultaneous gradients in your own code. Compare finite differences and known values before running large batches. Keep the supplied reference unchanged for comparison and identify every learner-written function in your report.

Apply: assemble a defensible report

• Corpus card: provenance, permission, split unit, shared generation assumptions and prior exposure.
• Frozen checkpoint and protocol: weights, source hashes, runtime, training settings and fixed sampling choices.
• Evaluation: nats per target, copy and position failures, count baseline, distinct valid samples and unsuccessful cases.
• Adaptation/retention: same starting model, declared rule changes, full versus adapter results and incompatible labels.
• Decision and next experiment: actual task requirements, costs omitted here, fallback behavior and genuinely new evaluation data.

The real-text comparison below starts the application component. The saved Project 5 workbench now collects snapshots and reports. Independent transfer assessment and human implementation review remain pending; wider task-based text evaluation remains incomplete. Learner pilots remain pending. These are remaining build requirements, not claims that a report or a green software test satisfies them.

Transfer the pipeline to real text

The extended package includes Lewis Carroll’s Alice’s Adventures in Wonderland from Project Gutenberg ebook 11, whose catalog lists public-domain status in the USA. The complete source and its notices are retained. The derived corpus selects three whole prose paragraphs from each chapter and collapses whitespace; it does not lowercase or strip punctuation. This is selected nineteenth-century English fiction, not representative coverage of language.

Chapters 1–8 supply 24 training paragraphs; chapters 9–12 supply 12 development paragraphs. Both are from the same public book. They are repeated development evidence, not an independent test. The character vocabulary is fitted only on training text; five development character occurrences map to unknown. End is scored; start is an input only. Context resets at every paragraph boundary.

node --experimental-strip-types real-text.mjs train 1000 real-checkpoint.json
node --experimental-strip-types real-text.mjs evaluate real-checkpoint.json real-results.json

The learned model has 825 parameters: a six-number embedding for each context character, a linear softmax head and output biases. It is a learned low-rank character bigram, not the earlier attention decoder; it does not transfer that synthetic checkpoint. Both this model and the add-one count baseline see only one previous character. Their limited context cannot capture long-distance dependencies.

Supplied 1,000-update result: training loss 2.5600 versus count baseline 2.4646; development loss 2.6097 versus count baseline 2.5482 nats per mapped target. The count baseline is better on both reported splits. A trainable model is not automatically an improvement. These are recorded results, not live training or comparable measurements to the synthetic decoder’s different vocabulary.

Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do: once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, “and what is the use of a book,” thought Alice “without pictures or conversations?”

ch1-p1: 303 targets including end; 0 unknown character occurrences. Learned loss 2.4259; count baseline 2.3746 nats per mapped target. Unknown mapping loses the identity of those characters; this is not raw-text likelihood.

atho tY _yureeisledind tolay ke arn ouge fencathe“if whe if

60 output tokens; end reached: yes. Four outputs use seed seven and no grammar repair. Inspect readability and failures; low next-character loss is not evidence of meaning or instruction following.

Explain one per-paragraph failure, compare learned versus counted context statistics, and propose genuinely new-source evaluation. Record preprocessing, unknown counts, fixed sampling and operating limits. The objective is measured token prediction; task quality and human judgment require separate evidence.

Original source and full notices · Derived paragraph corpus · Provenance and hashes · Supplied real-text checkpoint · Recorded evaluation and samples