Simpson's paradox

Simpson's paradox is a phenomenon in probability and statistics in which a trend appears in several groups of data but disappears or reverses when the groups are combined. This result is often encountered in social-science and medical-science statistics,^[1][2][3] and is particularly problematic when frequency data are unduly given causal interpretations.^[4] The paradox can be resolved when confounding variables and causal relations are appropriately addressed in the statistical modeling^[4][5] (e.g., through cluster analysis^[6]).

Simpson's paradox has been used to illustrate the kind of misleading results that the misuse of statistics can generate.^[7][8]

Edward H. Simpson first described this phenomenon in a technical paper in 1951,^[9] but the statisticians Karl Pearson (in 1899^[10]) and Udny Yule (in 1903^[11]) had mentioned similar effects earlier. The name Simpson's paradox was introduced by Colin R. Blyth in 1972.^[12] It is also referred to as Simpson's reversal, the Yule–Simpson effect, the amalgamation paradox, or the reversal paradox.^[13]

Mathematician Jordan Ellenberg argues that Simpson's paradox is misnamed as "there's no contradiction involved, just two different ways to think about the same data" and suggests that its lesson "isn't really to tell us which viewpoint to take but to insist that we keep both the parts and the whole in mind at once."^[14]