Learn with a generator and discriminator.

Make examples and learn to distinguish them

Imagine a generator that proposes two-coordinate points and a discriminator that tries to distinguish those proposals from a small reference dataset. A generative adversarial network, or GAN, trains these two models together. The generator does not receive a target point for each proposal. Instead, it receives a gradient through the discriminator’s current judgment.

Our reference has four groups, with 32 training and 32 distinct development points. The generator maps a random two-number Gaussian input z to a point G(z). Its 16 hidden tanh units and linear output contain 82 parameters. The discriminator reads a point and produces a score D(x) between zero and one through a sigmoid. Its 16 hidden units and single output contain 65 parameters. Both are real trained networks; the target centers are never used to snap generated points into place.

The discriminator’s job

The discriminator should score real training examples highly and generated examples low. Its loss uses binary cross-entropy. With N real points and N generated points, the lab minimizes:

Here xᵢ is a real training point and zᵢ is a random input. Logs are natural logarithms, so loss is measured in nats. The denominator averages the 2N classification examples. If one real example scores 0.8 and one fake example scores 0.3, their average loss is −(log 0.8 + log 0.7)/2, about 0.290. Confidently wrong predictions cost more.

The implementation uses an algebraically equivalent stable logit calculation, avoiding log zero when sigmoid scores round to endpoints. A discriminator score is about real-versus-generated classification under this training setup. It is not an independent factuality, safety or artistic-quality judgment.

The generator’s job

The original GAN framework describes a minimax game. This lab uses the common non-saturating generator objective, which encourages high discriminator scores without using the minimax generator loss:

A generated point scoring 0.2 contributes about 1.609; at 0.8 it contributes about 0.223. Minimizing this objective encourages the generator to produce examples that the current discriminator scores as real. The lab explicitly labels this objective rather than treating all GAN variants as identical.

The gradient travels backward through the discriminator to the generated coordinates, then through the generator to its parameters. The discriminator’s weights remain frozen during this generator update. The point inspector shows the gradient of the single-example generator loss with respect to the two generated coordinates. Negating that vector gives a local descent direction. Actual training changes shared generator weights, so it affects many generated points at once; it does not directly drag one displayed point.

Alternate the updates

Each cycle performs either one or five discriminator updates while the generator stays fixed. It then performs one generator update through the updated discriminator, with the discriminator now fixed. Within each update, gradients are computed from the same preceding parameters before any weight is changed. Both networks use Adam at rate 0.01, moment correction and norm clipping at five.

Fresh seeded random inputs are used for training. Fixed random inputs are reused for monitoring. Changing the update ratio restarts both networks from the same initial values, making comparisons easier to interpret. Five discriminator updates per cycle cost more than one; equal cycle counts are not equal training work. Controls cap calls at 25 cycles and the experiment at 2,000 cycles.

Why the loss curve can mislead

In ordinary supervised learning, a fixed evaluation objective makes a falling loss easier to interpret. Here the opponent changes. A stronger discriminator can raise generator loss even while generated examples improve. A weak discriminator can give flattering scores to poor samples. Even a score near one-half does not prove equilibrium: an uninformative discriminator can also output one-half.

Mode collapse means the generator concentrates on too few kinds of output. Our four target groups make this visible. Twenty-four points near one center can look individually plausible while missing three groups. Use the per-mode counts alongside proximity. The fixed-seed history lets you inspect temporary failure, recovery and changing losses without treating any one iteration as a guarantee.

Try the one-update ratio through 1,000 cycles and capture its output. Continue to 2,000 and compare. Then reset with five discriminator updates per cycle. These are exposed teaching cases, not a promise that a particular ratio is generally better. Repeat sampling seeds and report failures, including points outside the plot axes.

Compare a different mechanism honestly

The supplied diffusion reference uses the same training target but a 370-parameter denoiser trained for 1,000 updates. Its checkpoint is reproducible from the repository’s reference-generation script. Loading it does not mean you trained it. Sampling 24 points costs 768 denoiser evaluations, versus 24 generator evaluations for the GAN. Parameter counts, training updates, objectives, backward passes and sampling mechanisms differ, so raw call counts cannot establish equal compute or overall efficiency.

Both outputs are scored against the same four centers using radius 0.45. Count covered modes and failures as well as average distance. These checks do not measure the precise ring distribution, balanced frequencies, memorization or usefulness for another task. The supplied result is public comparison evidence, not an independent final test.

For a guided project, capture two GAN stages with the same seed and compare the diffusion reference. Explain why their losses and call counts cannot be directly ranked as a single quality score. For implementation practice, independently derive one discriminator weight gradient and one generator gradient through a frozen discriminator, then verify with central finite differences. For a decision brief, specify the missing-mode tolerance, failure handling and evidence needed before using a generator. These activities are practice; independent generative-project assessment and human review remain pending.

Return to diffusion training and sampling for its separate objective, or use the expanded catalog to explore the planned distribution-evaluation lesson.