Find unusual examples.

A flag asks for investigation

A machine reading looks different from earlier readings. That could mean a fault, a new but valid operating mode, a changed sensor convention, or a recording error. Anomaly detection scores unusualness relative to a reference. It does not supply the separate evidence needed to confirm a problem.

The lab uses synthetic machine measurements with teaching inspection labels. A reference cohort contains reviewed normal records. The detector fits only their coordinates. The development cases’ inspection results are available so you can count mistakes, but they do not fit the reference mean, spread or distance rule.

Start with two normal operating modes and the neighbor detector. Inspect a flagged case and explain what the flag actually establishes. Then add third-mode normal development cases without expanding the reference. Watch how many legitimate new cases become alerts.

Choose what “far away” means

The two detector choices fit the same coordinate means and spreads from reference records. Each coordinate gap is divided by its reference spread, so a coordinate with a larger numerical range does not automatically dominate.

For one reference mean, the query’s standardized coordinates are:

$z_x=\frac{x-\mu_x}{s_x},\qquad z_y=\frac{y-\mu_y}{s_y}$

x and y are the query measurements; μₓ and μᵧ are reference means; sₓ and sᵧ are reference spreads, calculated as the square root of the mean squared deviation. zₓ and zᵧ are unitless standardized coordinates. Means and spreads use only reference data, with the number of reference records as divisor. This lab rejects a reference with effectively zero spread instead of silently dividing by zero.

If the reference means are (0, 0) and both spreads are 1, a query (3, 4) has standardized coordinates (3, 4). Its distance to the global center is √(3² + 4²) = 5. A score of 5 is a distance, not a probability or a “five out of five” problem rating.

The neighbor detector instead finds the nearest k reference records under those standardized distances and averages the k distances. Here k is 1, 3 or 5. If the three nearest distances are 0.4, 0.7 and 1.0, the score is (0.4 + 0.7 + 1.0)/3 = 0.7. The selected-case table shows the actual neighbors and their distances; ties follow reference order.

This is a fitted distance detector, not a probabilistic density estimator. The global-center rule assumes one central region is useful. Neighbor distance can represent multiple populated regions, but still depends on reference coverage, scaling, k and the threshold.

A normal reference can have a misleading center

Normal modes A and B sit on opposite sides of the plot. Their global mean lies between them, where no reference records occur. Inspect problem case P1 at the gap. Its global-center distance is approximately zero, while its distance to nearby observed records is larger.

This is a limitation of describing several normal modes with one center. It does not prove the neighbor detector is universally better. Compare false alarms and misses on the same cases, and inspect individual explanations instead of selecting from one convenient example.

Changing reference coverage also changes its fitted means and spreads. The same numerical threshold after refitting may correspond to a different operating point. Record the reference configuration as part of the detector definition and reevaluate the threshold; do not compare score magnitudes from different fits as if their meaning were fixed.

Choose a threshold, then examine its consequences

The lab flags a case when its score is at least the threshold. Equality flags. Lowering the threshold can increase both detected problems and false alarms; raising it can reduce alerts while missing problems. Moving the threshold does not change any fitted score.

A false alarm is a flagged normal case. A miss is an unflagged confirmed problem. The problem-detection rate divides correctly flagged problems by all confirmed problems. The normal false-alarm rate divides flagged normal cases by all normal cases. Precision divides correctly flagged problems by all flags. Each has a different denominator; a rate with a zero denominator is undefined.

The fixed development cohort deliberately contains many problems to expose failure modes. Its problem fraction is not an estimated deployment prevalence. Repeatedly choosing settings from these cases uses development evidence; no untouched final test is supplied.

Unseen normal behavior needs evidence, not automatic rejection

Third-mode normal cases occupy a region missing from the default reference. They can therefore be distant and flagged even though their teaching inspections say normal. An unseen pattern is not automatically a faulty pattern.

The optional A/B/C reference includes separately reviewed, distinct records for the third mode. These are not the development cases being evaluated. Adding them models a deliberate review and reference update, not the rule “every flagged record should become normal.” Compare the third-mode false alarms and all problem misses after the update. Wider reference coverage can change sensitivity as well as specificity.

The normal mixture matters. A false-alarm rate measured mostly on familiar operation may fail when new operation becomes common. Before using the detector elsewhere, collect representative reviewed evidence and document what has changed.

Some problems need a different measurement

Normal case N1-5 and problem case P8 share the same two measurements. Their inspection labels differ because a separate inspection detected a hidden fault in P8. Any deterministic detector restricted to these measurements must give both the same score. A threshold cannot separate them.

Their overlap in the plot is intentional; select each ID to inspect its label and score. Possible responses include collecting another informative measurement or using an independent review process. More aggressive thresholds can create false alarms without resolving this missing-information problem.

Count rare events before interpreting alerts

Suppose 10,000 cases contain 1% true problems. There are 100 problems and 9,900 normal cases. If a detector flags 90% of problems and falsely flags 1% of normals, it produces:

• 100 × 0.90 = 90 true alerts.
• 9,900 × 0.01 = 99 false alerts.
• 90 / (90 + 99) ≈ 47.6% precision.

A 1% false-alarm rate does not mean only 1% of alerts are false. When problems are rare, normal cases supply a much larger denominator.

$\text{alert precision}=\frac{\pi r}{\pi r+(1-\pi)f}$

π (pi) is the assumed fraction of cases that are problems; r is the conditional problem-detection rate; f is the conditional normal false-alarm rate. All three are fractions between 0 and 1. Read the formula as “the expected true-alert fraction divided by the expected total-alert fraction.” If no alerts are expected, precision is undefined.

The lab’s deployment calculation reweights its observed conditional rates using an assumed problem rate. It does not change the detector or manufacture new observations. It assumes those rates—and the normal-mode mixture behind the false-alarm rate—remain applicable. The small, selected teaching cohorts cannot establish that for a real deployment. Expected counts can be fractional; they are not observed events. The sampling and uncertainty lesson explains why sample design and limited counts matter.

Turn a score into a defensible review policy

Record reference coverage, the detector, k, threshold, cohort counts and deployment assumptions. Estimate alert workload before deciding who will review it. Explain one false alarm, one miss and what an independent confirmation would involve. A low anomaly score should not be treated as proof that all relevant conditions are normal.

For implementation practice, reproduce reference-only scaling and nearest-distance scores, test threshold equality, and verify that changing the assumed prevalence does not refit the detector. For decision practice, specify how flags are reviewed and what evidence would justify a reference update. The export is a practice artifact, not independent assessment, human-reviewed deployment approval or evidence of learner effectiveness.