Find structure without labels.

Two different questions about the same points

You have measurements but no supplied categories. Clustering asks which points to group under a chosen distance rule. Dimensionality reduction asks how to represent points with fewer coordinates while retaining a chosen kind of information. A visualization displays a representation; it does not prove that its apparent groups are meaningful.

Start with the lab’s sixteen synthetic points. Eight lie just below the horizontal axis and eight just above it. They vary much more from left to right than from bottom to top. Would you group left versus right, or preserve which narrow band each point belongs to? The answer depends on what you want to do, not only on geometry.

K-means alternates two concrete actions

Choose k, the number of centers. The lab initializes them from a seeded selection of observations, then alternates:

1. Assign: send each point to its nearest center. Keep centers fixed during this step.
2. Move: replace each center by the coordinate-wise mean of its assigned points. Keep memberships fixed during this step.

For a separate worked example, take points (0, 0), (2, 0), (8, 0) and (10, 0), with initial centers (0, 0) and (10, 0). The first two join the left center; the other two join the right. Squared distances sum to 0 + 4 + 4 + 0 = 8. Moving the centers to (1, 0) and (9, 0) reduces that sum to 1 + 1 + 1 + 1 = 4. The memberships have not changed during the move.

$d^2=(x-c_x)^2+(y-c_y)^2$

x and y are one point’s working coordinates; cₓ and cᵧ are a center’s coordinates. d² is squared Euclidean distance: square the horizontal gap, square the vertical gap, and add them. For point (2, 1) and center (0, 0), d² = 4 + 1 = 5. Comparing squared distances gives the same nearest center as comparing their square roots.

The objective is the sum of these squared distances over stored group memberships. Assignment chooses a smallest distance; moving to the mean minimizes each group’s squared error. Each step therefore cannot increase this objective, apart from numerical rounding. That does not establish the globally best grouping: different initial centers can converge to different solutions.

After a center move, the plotted memberships still come from the preceding assignment. Some points might now be closer to another center. Press Assign points to nearest centers again to update them. This distinction matters when reading an intermediate trace.

Handle the cases a neat diagram can hide

Equal distances go to the lowest center number. An empty group keeps its old center instead of inventing a mean. This can leave fewer occupied groups than the chosen k. Identical observations can make initial centers coincide even when their row IDs differ.

The lab stops when centers no longer move within its numerical tolerance, or after twenty center updates. Budget exhaustion is not labelled convergence. Neither stopping condition establishes useful categories. Group numbers are arbitrary identifiers: C1 is not a verified species or a superior group.

Scaling changes what counts as close

A horizontal gap of 2 and vertical gap of 1 have squared distance 5. Multiplying the vertical coordinate by 10 changes that distance to 2² + 10² = 104. An innocent unit conversion or a deliberate feature weight can dominate a distance calculation if the rest of the pipeline does not account for it.

Every lab view subtracts the cohort mean. Translation does not change distances. With standardization, it also divides each coordinate by its own spread, calculated from this cohort:

$z=\frac{a-\mu}{s}$

a is the scaled coordinate, μ (mu) is its cohort mean, and s is the square root of the mean squared deviation from that mean. z is the standardized, unitless coordinate. These are descriptive cohort statistics with divisor n, the number of points. A mean of 6 and spread of 2 turn a value of 8 into z = (8 − 6)/2 = 1. If a coordinate is constant, its centered values are zero; the implementation uses divisor 1 rather than dividing by zero.

A positive multiplier applied before both mean and spread calculation cancels under this standardization. Try y multipliers 1 and 10 with standardization enabled. Without it, the geometry changes. Standardization itself is a modeling choice: it gives each coordinate equal marginal variance, not automatically the right importance for your task. Fit such transformations on the intended development/training cohort, then reuse them when evaluating new data.

The two plots use equal horizontal and vertical scales. Change their shared viewing bounds if points are clipped. Plot zoom changes the picture, not the fitted clusters or projection.

PCA keeps a high-variance direction

Principal component analysis (PCA) finds directions of large variation in centered data. In this two-coordinate lab, keeping the first principal axis reduces each point to one signed score along a line. It does not look at the k-means groups or use category labels.

Take a unit direction u = (0.6, 0.8), whose squared components add to 1. A centered point (3, 4) has score 3 × 0.6 + 4 × 0.8 = 5. Reconstructing from that score gives 5 × (0.6, 0.8) = (3, 4): this point lies exactly on the axis.

A nearby point (3, 0) has score 1.8. Its reconstruction is (1.08, 1.44), so the perpendicular difference has squared length (3 − 1.08)² + (0 − 1.44)² = 5.76. That detail is discarded when only the score remains.

$t=(x-\mu_x)u_x+(y-\mu_y)u_y,\qquad \hat{\boldsymbol{x}}=\boldsymbol{\mu}+t\boldsymbol{u}$

t is the retained score; μₓ and μᵧ are the two means; uₓ and uᵧ are the components of the unit axis direction. The bold symbols group two coordinates: μ is the mean point, u is the direction, and x̂ is the reconstructed point. Read the expressions as “measure the centered point along the chosen direction, then move that far along the direction from the mean.” Coordinates and scores use the working units; squared reconstruction error uses their squares.

The selected point, its reconstructed square, the connecting discarded-distance line and the numerical trace describe the same calculation. Changing to a manual axis tests what a different direction would preserve. Neither a manual axis nor PCA changes the original observations.

Retained variance is not task accuracy

In the default synthetic cohort, x variance is 5.25 and y variance is 0.16. The first principal axis is horizontal. Its retained fraction is 5.25 / (5.25 + 0.16), approximately 97.04%. Yet B1 and B9—and every corresponding lower/upper pair—receive the same score. All information about which band a point occupied disappears.

That small discarded variation could be crucial to a task. A 97% retained-variance figure is not 97% classification accuracy, a probability of correctness, or proof of usefulness. Change the axis to 90° to preserve band separation; observe the different variance fraction and lost horizontal detail.

Multiplying y by 10 changes its variance by 100, because variance squares deviations. Its variance becomes 16, so the leading axis becomes vertical. Standardizing both coordinates instead gives equal variances here: the first direction is no longer unique. The implementation chooses the x direction to make this tied case reproducible and labels the tie explicitly. If every point coincides, total variance is zero and retained fraction is undefined.

Optional: how the two-coordinate principal axis is calculated

The implementation computes mean centered products: a = mean[(x − μₓ)²], b = mean[(x − μₓ)(y − μᵧ)], and d = mean[(y − μᵧ)²]. Here a and d are coordinate variances and b is covariance, which describes whether the coordinates tend to move together. The covariance matrix is the two-by-two arrangement with a and d on the diagonal and b in both off-diagonal positions.

Its principal variances are λ₁,₂ = (a + d ± √((a − d)² + 4b²))/2. The λ symbols are the variances along two perpendicular principal directions; the larger one identifies the first axis. The lab calculates these values and displays them beside the retained fraction. No matrix manipulation is required to run the guided experiment. Small numerical ties are treated as nonunique directions, not as a meaningful preference for x.

Explore real measurements without inventing labels

The Iris option reuses 98 original, training-side representatives of identical four-measurement groups. x is sepal length and y is petal length; both are features in this lesson. Their original units are cm. Grouping identical measurements does not authenticate individual specimens or establish independent sampling. Species labels and the separate 51-row validation cohort are not passed to k-means or PCA.

The local source manifest records Fisher (1936), UCI, CC BY 4.0, the source size and hash. These measurements are unchanged. Any clusters are exploratory geometry; do not call them recovered species. This lab supplies no held-out estimate of future clustering usefulness.

For an implementation exercise, reproduce the assignment/mean steps, empty-center behavior and perpendicular reconstruction check. For a decision exercise, define a useful grouping purpose and describe independent evidence that would validate it. Export the settings, objective, projection and a note explaining one lost distinction. These are practice artifacts; independent project assessment, human review and learner pilots remain separate.