Generate by learning a flow.

Move a cloud of points

Imagine moving a scattered cloud into four groups. Instead of predicting a final destination in one jump, a model tells each point which direction to move at its current position and time. Repeated small moves generate a new cloud. Try an untrained field first, then train and compare the result using the same starting points.

This lab uses the diffusion lesson’s original synthetic dataset: 32 training points and 32 distinct public development points around four centers. Its separate neural network reads two coordinates and time and predicts two velocity coordinates. Two hidden layers of sixteen tanh units give 370 parameters. No image model, external service or GPU is required.

Construct a training path

Draw a two-coordinate source z from a standard Gaussian: each coordinate has mean zero and variance one. Independently choose a training target x. Time t runs from zero to one; it is normalized path time, not seconds on your computer. A straight path connects the pair:

xₜ is the intermediate position; u is the target velocity, in coordinate units per unit of path time. Read this as “start with a fraction of the source and a fraction of the destination.” With z = [−0.6, 0.8], x = [1.2, −1.2] and t = 0.5, the position is [0.3, −0.2]. The velocity is [1.8, −2.0]. Over half a unit of time that velocity moves the source by [0.9, −1.0]. Change the inspector’s time and check both endpoints yourself.

These independent pairs are not an optimal-transport matching. Different pairs can pass through similar positions with different velocities. The model sees only position and time, not the pair’s hidden destination. Squared-error training encourages a conditional average of the possible velocities there. Straight training-pair paths therefore do not imply straight generated trajectories.

Learn the field

Write vθ(xₜ, t) for the network’s predicted velocity; θ denotes its parameters. For N two-dimensional examples the loss is:

The squared length means “square the error in each coordinate and add.” Dividing by 2N averages across examples and their two coordinates. A prediction [1, −1] against target [2, −3] contributes (1 + 4)/2 = 2.5 to a one-example loss. This is velocity error, not exact likelihood and not the diffusion lesson’s noise error; those loss numbers cannot rank the two generators directly.

Each update creates fresh source points and times for the 32 training targets. Adam uses simultaneous gradients, moment correction and gradient-norm clipping at five. Buttons run at most 25 updates, with a total limit of 1,000. Monitoring reuses fixed pairs at five times, including endpoints, and compares the model against zero velocity on those same pairs. Development monitoring is repeated practice evidence, not an untouched qualification test.

Generate with small numerical steps

Freeze the model and draw 24 Gaussian starting points. The sampler never reads their intended destination: none was assigned. Let h = 1/S be a step size for S steps. Euler’s rule reads the current velocity and moves in that direction:

For position [1, 2], velocity [−0.4, 0.8] and h = 0.25, Euler produces [0.9, 2.2]. The field may change during this move. Heun’s method first makes this Euler prediction, reads the velocity there at the new time, then uses the average of the two velocities for the actual move. If the second velocity is [−0.2, 0.4], Heun produces [0.925, 2.15]. The lab shows the first generated point’s actual arithmetic for every saved step.

Euler uses one field evaluation per point per step; Heun uses two. With 24 points, Euler with 32 steps and Heun with 16 steps each use 768 individual field evaluations. Equal calls do not prove equal elapsed time: bookkeeping and rendering also take work. Inspecting an already saved frame performs no new field evaluations.

There are two different error sources: the learned field may be inaccurate, and a finite-step integrator approximates motion through that field. Smaller steps can reduce integration error without repairing model error. More steps need not improve our sample-quality scores. No new noise is added after initialization in this sampler; different starting seeds still produce different samples.

Compare evidence and limits

Train one field, then compare Euler with 4 and 32 steps using seed 7. Capture both finished runs. Next compare Euler 32 with Heun 16 for equal field-call budgets. Use another seed before generalizing. Frozen runs retain their original model even if you continue training the live model. Fixed plot axes show out-of-range counts rather than hiding failures by clipping coordinates.

The final summary reports distance to the nearest center and counts within radius 0.45 of each center. Check coverage as well as total proximity. These diagnostics do not establish the target ring shape, correct frequencies or full distribution agreement. A successful toy sample is not evidence of image-generation capability.

For a guided project, export two captures and explain one quality/work tradeoff with numbers. For implementation practice, independently implement Euler and Heun on the field dx/dt = x, whose solution from x = 1 is e at time one; compare error at equal field-call budgets before applying the methods to a neural field. For a decision brief, select a budget, report failure counts across seeds, and state what evidence would change your choice. These are practice tasks; exported notes and captures are not independent assessment or human-reviewed implementation evidence.

Continue with the diffusion experiment to compare mechanisms on the same target. The expanded catalog shows the remaining generation topics. Learner pilots and human artifact review remain pending.