Ask what would change under intervention.

Compare like populations

In our fictional training program, half the population has low support need and half high need. Without the program, their pass rates are 70% and 10%. With the program, they are 90% and 30%. The program raises both rates by 20 percentage points.

Yet only 10% of low-need people enroll, compared with 90% of high-need people. Among participants, 90% have high need, so their pass rate is 0.1×0.9 + 0.9×0.3 = 36%. Among nonparticipants, 90% have low need, so their pass rate is 0.9×0.7 + 0.1×0.1 = 64%.

The observed difference is −28 percentage points, despite a positive effect in each group. The two participant groups are not comparable populations. This is a confounded association, an example of an aggregate reversal. These numbers are synthetic teaching assumptions, not findings about a real program.

Put the assumptions into a graph

Let Z mean support need, T program assignment and Y passing. A directed arrow represents an assumed causal influence. Z → T and Z → Y create the path T ← Z → Y: a common cause affects both assignment and outcome. T → Y represents the program’s effect.

Hiding Z from the dataset does not remove its influence. Removing T → Y is a stronger change: in this simulator the program then changes no individual’s outcome. Randomizing assignment removes Z → T while retaining the need-to-outcome relationship.

The graph is supplied, not discovered from correlation. Drawing an arrow does not establish that it exists in the world. Causal reasoning must state how the graph is justified and what important causes may be missing.

Conditioning and intervention answer different questions

P(Y=1 | T=1) asks what fraction pass among those assigned the program. P(Y=1 | do(T=1)) asks what fraction would pass if we set program status for the target population, replacing the original assignment mechanism. The notation “do” marks that replacement.

Our target population stays half low need and half high need under either intervention. Set everyone to program and the pass rate is (0.9+0.3)/2 = 0.6. Set everyone to no program and it is (0.7+0.1)/2 = 0.4. Their difference is 0.2, or 20 percentage points. Do not confuse that with a 20% relative increase: the relative increase from 0.4 to 0.6 would be 50%.

The app knows these mechanisms because it created the synthetic world. Showing its exact intervention answer does not prove that an analyst could recover that answer from the observed data alone.

Adjustment requires information and overlap

Under this graph, recording need allows an analyst to compare program statuses within each need group, then combine those comparisons using the target population weights. With no additional confounding, stable intervention meaning and both statuses represented in each group, the adjustment formula is:

Here t is the chosen program status, z ranges over low and high need, and P(Z=z) is the target group’s weight. Read this as: estimate each need group’s pass rate under the chosen status, then average over the same target population. This does not use the different need distribution within the enrolled group.

If need is unrecorded in the biased assignment case, the necessary conditional rates cannot be formed. The effect is not identified from those variables alone without additional assumptions. “Identified” means the available distribution and stated causal assumptions determine the target quantity; it does not mean a finite estimate has no uncertainty.

If low-need people never enroll and high-need people always enroll, need is recorded but overlap, also called positivity, fails. We have no observed low-need participants or high-need nonparticipants. More observations under that same rule do not fill those cells. Supplying their outcomes from the simulator would be an extra model assumption, not empirical adjustment.

Adjustment is not a rule to control every available column. Variables caused by the intervention or selected through other causal structures may require different reasoning. The current lesson implements one explicit common-cause graph, not a general causal-identification solver.

Randomization changes the assignment process

The randomized study assigns the program independently with 50% probability to each person. It breaks the common-cause link into assignment, so the population comparison identifies the intervention effect under this simulator’s assumptions even without measuring need.

Finite groups will not be perfectly balanced. Generate 200 people per study, then 2,000, and change the public seed. Inspect counts before comparing percentages; an empty group has an undefined rate, not zero. The app reports sample differences without a confidence interval or significance claim.

Randomization alone does not fix noncompliance, differential dropout, measurement error or interference between participants. This simulator excludes those problems and assumes a stable program. A real study needs a protocol addressing them. Reusing the public synthetic examples is practice, not independent evidence that a real intervention works.

A counterfactual asks about the same person’s alternative

Focus on high-need people. The pass rate without the program is 10%, and with the program 30%. Among those who would pass with it, what fraction would also pass without it?

A shared-rank model assumes favorable individual factors align across the two alternatives. All 10% who would pass without are inside the 30% who would pass with. The answer is 10/30 = one third.

An opposed-rank model can make those two success groups disjoint. Then none of the program passers would pass without, giving zero. Both models preserve the 10% and 30% marginal rates and therefore the same observed and randomized population tables. They differ in how the two potential outcomes pair within an individual.

In the simulator, a uniform latent number U represents individual response factors. Shared rank compares the same U with each pass threshold. Opposed rank compares U for no program but 1−U for program. Each has uniform marginal noise, while the paired outcomes differ. With the program edge removed, the no-individual-effect assumption forces identical outcomes instead; the opposed pairing is disabled.

Given only two marginal success probabilities p₀ and p₁, the probability of succeeding under both alternatives can lie between max(0,p₀+p₁−1) and min(p₀,p₁). Dividing by p₁, when positive, gives bounds among those who would pass with the program. For high need, that yields zero through one third. Selecting a particular value requires more than the two population rates.

The joint table is not a directly observed record of giving each person both mutually exclusive assignments. It expresses structural assumptions. A randomized average effect is valuable, but it does not uniquely reveal every individual’s counterfactual outcome.

Write a claim that matches the evidence

For guided practice, explain the 36% versus 64% reversal and repair it using the common target population. For implementation practice, reproduce the group counts and independent population arithmetic, preserving undefined empty cells. For a decision brief, state the intervention, target population, graph, measurement, overlap and missing-outcome assumptions before recommending a study.

Try a different case: two equally sized need groups have untreated rates 60% and 20%, and treated rates 80% and 40%. The treated group contains 25% low need, while the untreated group contains 75% low need. Compute the observed difference and the same-population intervention effect.

Check this practice case

The treated rate is 0.25×0.8 + 0.75×0.4 = 0.5. The untreated rate is 0.75×0.6 + 0.25×0.2 = 0.5. The observed difference is zero. Under the common 50/50 target population, treated is 0.6 and untreated 0.4, an effect of 20 percentage points. This is explained practice, not an independent assessment.

The temporary audit exports assumptions and simulations. A graph, a completed note and a software check do not establish a real causal effect or learner understanding. Independent project review and learner pilots remain pending.