Predict a class probability.

Predict a chance before choosing an action

A system might estimate a 70% chance that a request needs attention. That estimate is different from deciding to act. A team with costly misses may act at a lower threshold; another with limited review capacity may need a different policy. First learn how a probability estimate is fitted, then inspect the consequences of its action rule.

This lesson uses two unitless numerical measurements and synthetic binary labels. Binary means the label is either 0 or 1. No real service outcomes are implied. The fixed training and validation inputs are disjoint, but every visit to validation is development practice.

Keep the weighted sum, change the output

Linear regression predicts an unrestricted number. A class probability should lie between 0 and 1. Logistic regression first calculates a weighted sum, called a logit, then applies a sigmoid to map that sum into a probability estimate.

For example, weights 1 and −1, inputs 2 and 1, and bias 0 produce z = 1 × 2 − 1 × 1 + 0 = 1. The sigmoid converts 1 into approximately 0.731. That means an estimated 73.1% chance of label 1 under the model, not a guarantee that this particular outcome will be 1.

$z=w_1x_1+w_2x_2+b,\qquad p=\frac{1}{1+\exp(-z)}$

x₁ and x₂ are the two input measurements; w₁ and w₂ are their learned weights; b is the learned bias; and z is the weighted sum. p is the estimated probability of label 1. The inputs in this lab are unitless, so the weighted sum is also unitless. exp is the exponential function with base e, approximately 2.718. The calculator in the workbench evaluates it for you.

At z = 0, exp(0) = 1, so p = 1 / 2 = 0.5. Positive logits give p above 0.5; negative logits give p below 0.5. Logistic regression is linear in its logit, even though its probability output bends smoothly. For two features, a fixed non-endpoint threshold gives a straight boundary when at least one weight is nonzero.

Fit probabilities with cross-entropy

A good probability-fitting objective should penalize confident wrong predictions more than uncertain ones. Binary cross-entropy uses the probability assigned to the observed outcome. If the true label is 1, predicting p = 0.8 gives loss −ln(0.8) ≈ 0.223; predicting p = 0.2 gives loss −ln(0.2) ≈ 1.609. If the true label is 0, the relevant probability is 1 − p instead.

$L=-\frac{1}{n}\sum_{i=1}^{n}\left[y_i\ln p_i+(1-y_i)\ln(1-p_i)\right]$

n is the number of training examples. For row i, yᵢ is its observed 0/1 label and pᵢ is its predicted probability of label 1. ln is the natural logarithm, the inverse of exp. The expression selects the log probability of the observed class, negates it and averages over examples. Loss is dimensionless; with natural logarithms it can be expressed in nats. You do not need to calculate logarithms by hand to run the guided experiment.

The implementation evaluates an equivalent stable expression directly from the logit. It does not take log(0) when a displayed sigmoid probability rounds to zero or one. Display precision and mathematical certainty are different things.

Change all parameters from the same model

Gradient descent changes parameters to reduce training loss. The learning rate sets the size of a step. For a single row, each weight’s contribution is (p − y) × its input; the bias contribution is p − y. Average these contributions across the training set.

For a worked example, start with zero weights and bias. With inputs (2, −1) and label 1, p = 0.5, so p − y = −0.5. The three gradient contributions are −1, +0.5 and −0.5. At learning rate 0.1, subtracting rate × gradient gives weight changes +0.1 and −0.05, and bias change +0.05. All contributions use the original zero-parameter model before any update is applied.

The lab’s one-CPU-update control uses the average over all 64 training rows, so it need not equal that one-row worked example. The optional update table shows the actual averages. The 100-update button performs the same simultaneous updates in a worker. Training is bounded at 1,000 accepted updates; cancellation retains the preceding accepted model.

Compare frequencies, not individual promises

A probability estimate of 0.8 does not promise an individual success. Calibration asks whether predictions near 0.8 are accompanied by about 80% positive outcomes over suitable repeated cases. Group the validation predictions into intervals, then compare each bin’s mean estimate with its observed positive fraction.

If five predictions average 0.8 and three observed outcomes are positive, the measured fraction is 3 / 5 = 0.6. That mismatch deserves investigation, but five cases cannot establish a stable calibration error. Empty bins have no observed fraction; they are not evidence of zero probability. Wider bins can also hide differences within them.

The lab’s plot uses identical zero-to-one axes and a diagonal showing equality of those two summaries. Every dot has a corresponding row with counts. The diagonal is not a confidence interval. Repeatedly choosing a model using these bins makes them development evidence; use appropriate new data for a final calibration claim.

Choose a threshold with consequences in mind

Predict label 1 when p is at least the selected threshold. Equality counts as positive. Moving that threshold changes the confusion counts, precision, recall and assumed error costs. It does not change weights, probability estimates, cross-entropy or the probability-bin table.

Try this before training: zero parameters give p = 0.5 everywhere. At threshold 0.5 every case is positive; at 0.55 every case is negative. The probabilities are identical in both experiments. This separates fitting the estimate from deciding how to use it.

After training, compare thresholds under different costs. A low numerical cost under the lab’s fixed assumptions is not automatically a deployable policy. Capacity, affected users, changing populations and unreliable cost estimates may matter. Use the decision-threshold lesson and cumulative decision project to develop that reasoning.

Know what the synthetic demonstration establishes

Training and validation labels are fixed seeded draws from sigmoid(−0.2 + 1.3x₁ − 1.1x₂). The trainer receives sampled training labels, not the generating probabilities or validation labels. Finite samples produce uncertainty even though the generating model belongs to this model family. This example does not show that real relationships are linear, that fitted probabilities are automatically calibrated, or that classroom performance transfers to a real service.

Use the downloaded experiment to explain a changed probability, an unchanged probability after a threshold move, a calibration-bin count and a limitation. Engineering and decision prompts are practice; independently reviewed artifacts and learner pilots remain pending.