Blinding isn't bureaucratic box-ticking — it's what keeps a trial's false-positive rate where it belongs. This simulation runs thousands of trials of a drug that secretly does nothing, and shows how quickly a broken blind turns it into a "success." Drag the sliders and watch the damage.
A Monte Carlo simulation run live in your browser: two arms (drug vs placebo), a continuous outcome in standard-deviation units, ~2,000 trials per setting. Unmasking adds a measurement bias to the drug arm proportional to how unblinded the trial is. Significance is a two-sided test at 5%. This models bias in a subjective endpoint — see the caveats below.
A first-draft write-up — the simulation is real; the words are a starting point to make your own. This one's your field, so treat my framing as a straw man to improve.
Every intro-to-trials course says the same thing: blind the trial. Patients shouldn't know what they're taking, and — just as importantly — the people measuring the outcome shouldn't know either. It's easy to file that under "good practice" without feeling why it matters. This simulation is an attempt to make it felt.
Set true drug effect to zero. Now the drug is a dud — no better than placebo. In a perfectly run trial, it should clear the significance bar about 5% of the time, purely by chance. That 5% is the false-positive rate we agree to live with.
Now drag unmasking up. Each notch assumes the outcome assessors are a little more aware of who's on the drug — and, being human, they rate those patients a little more generously. The biology hasn't changed. Only the measurement has. Watch the damage curve climb off the 5% line.
The second chart is the real mechanism. Each simulated trial produces one estimated effect; across thousands of trials those estimates form a distribution. Perfectly blinded, that distribution sits centred on zero, and only its thin tail pokes past the significance threshold — that's your 5%.
Unmasking doesn't widen the distribution. It slides the whole thing to the right. A bias that looks modest — a few tenths of a standard deviation — marches a large share of the distribution across the line. The trial isn't noisier; it's aimed wrong, and no sample size fixes a target that's off-centre.
Bigger trials don't rescue you here. More patients shrink the noise but not the bias — so a large, unblinded trial is a precise measurement of the wrong number.
Turn the sample size up with a nonzero bias in play. The false-positive rate doesn't fall — it often rises, because a larger trial is better at detecting the bias as if it were a real effect. Precision without validity is a trap, and it's one that a bias like this walks you straight into.
Since this is your field, here's precisely what the simulation does — the trial it assumes, the equations each patient is drawn from, and which knob moves which symbol. Nothing is hidden: it's a few lines of arithmetic run thousands of times.
Every patient gets one outcome Y, drawn from a normal distribution. Placebo patients centre on zero; drug patients centre on the true effect plus a bias:
δ = the true drug effect · β = the largest assessor bias, reached at full unmasking · u = the unmasking fraction, from 0 (perfectly blinded) to 1 (fully unblinded). The one term that does all the damage is β·u — a bias added to the drug arm's measured mean, growing linearly as the blind fails.
For each trial the code draws n patients per arm, then runs the ordinary two-sample comparison:
M and s² are each arm's sample mean and sample variance (the usual n−1 form); the standard error is the unpooled (Welch) version. The cutoff 1.96 is the two-sided 5% normal critical value — a z-test; at n ≥ 20 the t-correction is negligible. Run this 2,000× and the share that clears the bar is the significance rate on screen: with δ = 0 it's the false-positive rate (it should be 5%); with δ > 0 it's statistical power. The damage curve repeats it 800× at each unmasking level.
| Slider | Symbol | Where it enters | What it does |
|---|---|---|---|
| True drug effect | δ | the drug-arm mean, N(δ + β·u, 1) | Set to 0 to study false positives; above 0 you're watching power instead. |
| Unmasking (blinding failure) | u | only inside the bias term β·u | Slides the whole sampling distribution to the right; moves the white marker along the damage curve. |
| Assessor bias when unmasked | β | the slope of bias vs. unmasking (its max, at u = 1) | How much a fully broken blind is worth, in SD. β·u is the bias actually applied. |
| Sample size per arm | n | the SE, √(2/n), and the red threshold √(2/n)·1.96 | Shrinks the noise but not the bias — so more patients make a biased trial worse. |
Because the bias lands on the mean and not the variance, the distribution translates rather than widens — exactly what the second chart shows. And the significance rate has a clean closed form that the Monte Carlo simply confirms:
Φ is the standard-normal CDF. With δ = 0 and no bias, both terms are Φ(−1.96) = 2.5%, summing to the 5% you're promised. Switch on a bias and the first term climbs — and crucially n sits inside it, as √(n/2), multiplying the bias. That's the sample-size paradox in one line: growing n makes β·u·√(n/2) larger, pushing the false-positive rate toward 100%. A bigger trial measures the wrong number more precisely.
This is a deliberately simple model, and it's built to make a point, not to be a trial-planning tool. A few things worth stating plainly:
Even with those simplifications, the qualitative lesson is robust and, I think, worth internalizing: blinding is not a formality layered on top of a good trial. For a subjective endpoint, it's part of what makes the trial's headline number mean anything at all.
Blinding, endpoints, power — the choices that decide whether your result means anything happen before data collection. That's exactly where I can help.