Understand Neural NetworksTHE BUILDING BLOCKS

Your first neuron.

Every neural network starts somewhere. Meet its smallest building block—and teach it to make a better prediction.

About 15 minutesInteractive lessonBeginner friendly

01TRY SOMETHING SIMPLE

Can a number learn a pattern?

A shop sells identical items. The total price depends on how many you buy. Our model makes a guess using two numbers: a weight and a bias.

\text{prediction} = \underbrace{w}_{\text{weight}} \times \underbrace{x}_{\text{input}} + \underbrace{b}_{\text{bias}}

A little rusty on equations? Take a quick refresher and come right back.

02MAKE A PREDICTION

What happens if we increase the weight?

Keep the input at 2 and the bias at 0. Before touching the sliders, take a guess.

03LEARN WITH YOUR HANDS

Make the neuron your own.

Open standalone

Move the weight and bias sliders. Then try One step to see how training changes them, or Watch it learn to run 150 small updates.

THE NEURON PLAYGROUNDLive · on your device

INPUT x2items

× 1.00

weight

NEURON+ 0.00bias

PREDICTION ŷ2.00credits

2 × 1.00 + 0.00 = 2.00

Input x2

0 items4 items

Weight w1.00

Bias b0.00

−25

Weight tilts the line.
Bias shifts it up or down.

Predictions meet examples

ModelExamples

MEAN SQUARED ERROR54.00

LEARNING STEPS0

LEARNING RATE0.02

Saved on this device

The dots are example prices. The line is your model. Dashed gaps show its errors. All calculations run in your browser.

04CONNECT THE DOTS

A small calculation with a big job

Imagine a shop where every item costs the same amount, plus a fixed packing fee. A simple model can estimate the total price from the number of items.

It needs just two adjustable numbers: a weight for the price per item, and a bias for the fixed fee. Together, they form our first neuron.

This is a made-up price example designed to make every calculation visible. Real prices depend on many more things.

What is actually learning?

In the playground, the four example prices are 5, 8, 11, and 14 credits for one through four items. The model begins with a poor guess: a weight of 1 and a bias of 0.

Training compares its predictions with the examples. It measures the errors, calculates how the weight and bias affect those errors, and adjusts both a little. Repeating this process is gradient descent.

Weight changes the slope. A larger weight adds more to the prediction for each extra item.
Bias changes the starting point. It raises or lowers every prediction by the same amount.
Loss measures the mistakes. Here, we average the squared errors across all four examples. Large mistakes count more.

One neuron is a starting point

This lesson uses a neuron with a linear output: its graph is a straight line. Later, we’ll add nonlinear activation functions and connect neurons into layers, so the network can represent more complicated patterns.

A perfect fit to these four examples only tells us the model fits this tiny training set. We still need new examples to check generalization.

MODEL PREDICTION2.00

EXAMPLE PRICE8.00

SIGNED ERROR-6.00

SQUARE IT(-6.00)² = 36.00

The model predicts 6.00 credits too low. Squaring makes both negative and positive errors nonnegative. Doubling an error makes its square four times as large.

AVERAGE ALL FOUR SQUARES216.00 ÷ 4 = 54.00

This is the playground’s mean squared error. All four examples count, not only the one selected.

Go deeper: see one learning update

For each example, calculate prediction minus target. The weight gradient averages twice that error times the input; the bias gradient averages twice the error. Both updates use the same old parameters.

w_{\text{new}} = w - 0.02\,\frac{\partial L}{\partial w}

b_{\text{new}} = b - 0.02\,\frac{\partial L}{\partial b}

At weight 1 and bias 0, the gradients are −40 and −14. One step gives weight 1.8 and bias 0.28. The average squared error falls from 54 to 24.0784.

05YOUR TURN

Check your understanding

A new prediction. Same idea.

Set aside the playground values. A model has weight = 3 and bias = 2. What will it predict for 3 items?

Predicted pricecredits

What does the bias do?Changes the extra price per itemAdds the same fixed amount to every predictionMakes the prediction random

06TAKE IT WITH YOU

You’ve met the smallest building block.

A neuron combines inputs with a weight and a bias. Learning adjusts those parameters to reduce error. A network connects many of these computations.

View my work

NEXT IN THIS MODULE

Loss functions & gradient descent

Why some updates improve a model—and others go too far.

Open

Adapted from ebook lessons 1–2.Read the source