Rebuttal When it comes to scientific inference, sometimes a cigar is just a cigar

We are grateful to the editors for suggesting that our submission become a debate piece, as we value critical discussion. We are gratified that the three invited counterpoints not only agree with our position but add useful insights. Elwood summed up our view when (referring to the White Paper on the U.S. National Children’s Study1) he commented that ‘the concept of external validity given confuses statistical inference with scientific inference’.2 Richiardi et al. echoed our point that representativeness is not desirable even if the goal is to study effect-measure modification: ‘Similarly using non-representative samples may enhance our ability to assess heterogeneity with regards to potential effect modifiers, e.g. by ensuring that there are adequate numbers in each of the ethnic groups to be considered if we suspect or are interested in potential modification by ethnicity’.3 And we especially liked Nohr and Olsen’s quotable remark that ‘Representativeness is time and place specific and will therefore always be a historical concept … .’4

Richiardi et al. suggested that ‘Perhaps Rothman and colleagues go too far in arguing that representativeness should be avoided as a matter of principle, and we consider that there are some situations where representativeness is the most sensible approach. For example, it would be rare for researchers to only study one age-group, and to then attempt to extrapolate their findings to other age-groups, if sufficient numbers and funding were available to also sample adequate numbers from these other age-groups’. But we in fact acknowledged that there is a role for representativeness is certain circumstances, as when ‘public-health professionals may rely on representative samples to describe the health status of specific populations’.5 Nevertheless, when studying effects across a range of a variable such as age, representativeness is not the most effective way to do so, …

Rothman, K., Gallacher, JEJ., & Hatch, EE. (2013). Rebuttal: When it comes to scientific inference, sometimes a cigar is just a cigar. International Journal of Epidemiology, 42(4), 1026-1028. DOI: 10.1093/ije/dyt124