Designing for cause

An experiment imposes treatments, the levels of an explanatory variable (or combinations of levels when there are several factors), on experimental units, called subjects or participants when they are people, and then measures a response variable on each unit. A well-designed experiment includes:

1. Comparison of at least two treatment groups, one of which may be a control group receiving no active treatment, a placebo, or the standard treatment.
2. Random assignment of treatments to experimental units.
3. Replication: more than one unit per treatment, so a difference between groups can be distinguished from the quirks of a single unit.
4. Direct control: holding potential extraneous sources of variation, the same soil, the same time of day, the same measurement protocol, constant across all units.

Note what replication means inside one experiment: multiple units per treatment, not running the whole study again next year. And note that a completely randomized design does not require equal group sizes; it requires that chance, and nothing else, decides who gets what.

An extraneous variable is one that may affect the response but is not the explanatory variable under study: fitness in a caffeine trial, sunlight in a fertilizer trial. The purpose of random assignment is to create treatment groups that are as similar as possible with respect to every extraneous variable, measured or not, named or not. When the coin flips decide group membership, fit and unfit, sunny and shaded, anxious and calm all get scattered across the groups in roughly equal proportion.

That scattering is what breaks confounding. A confounding variable is related to the explanatory variable in a way that makes it impossible to tell which of the two is moving the response. Give method A to the morning class and method B to the afternoon class, and time of day travels with the method: alertness is now an alternative explanation, and no analysis afterward can untangle it. Randomly assign methods within the school day and time of day no longer tracks the treatment; it has become noise instead of an explanation.

Keep the two randomnesses in their lanes. Random assignment is what supports a cause-and-effect conclusion. Random selection is what supports generalizing to a population, and an experiment can be causal without it: units are often volunteers, because randomly conscripting people into treatments is usually impossible or unethical. The price of volunteers is scope, not causality: the conclusion applies to units like the ones studied.

People given something respond to the giving. The placebo effect is the difference between the average response to a placebo, an inactive treatment, and the average response to no treatment at all. It is a real effect on real responses, which is exactly why the control group in a medical trial gets a placebo rather than nothing: both groups then carry the same expectation, and the comparison isolates the drug's chemistry from its ceremony.

Blinding controls whose expectations can leak into the data. In a single-blind experiment, either the participants do not know which treatment they are receiving, or the members of the research team who interact with them and measure the response do not, but not both. In a double-blind experiment, neither side knows. The second blind matters because measurement is human too: an evaluator who knows which patients got the new drug scores ambiguous symptoms differently, without ever intending to.

Which blind to demand depends on the response variable. A blood assay read by a machine resists expectation; a pain rating, a skin assessment, a behavior tally all bend toward what someone hopes to see, and those call for blinding on both sides of the clipboard.

Random assignment balances extraneous variables on average. When one of them is known in advance to drive the response, you can do better than average: block on it. In a randomized block design, units are first grouped into blocks of similar units, by prior fitness, by field, by starting weight, and treatments are randomly assigned within each block, so every treatment appears in every block. The variation the blocking variable causes is then separated from the treatment comparison, which is made between similar units and comes out more precise.

A matched pairs design is blocking taken to its limit: blocks of two, matched on extraneous variables, with the two treatments randomly split within each pair. Or the pair is one unit playing both roles, receiving both treatments in a random order. Either way, the comparison happens inside the pair, where almost everything extraneous is identical.

Blocking is the experiment-side twin of stratifying, and the exam expects you to keep them straight: stratifying organizes selection from a population in a sampling design; blocking organizes assignment of treatments in an experiment. "We recruited equal numbers of men and women" is a sampling decision. "We randomly assigned treatments within the men and within the women" is blocking. One more distinction and the topic closes: choose blocking variables because they affect the response. A block built on an irrelevant variable, like blocking a typing study by eye color, adds structure and buys no precision.