Commentary: On representativeness

Most epidemiological studies—indeed, all the interesting ones—are designed to assess a potential causal relationship. There are often difficult choices in the selection of the subjects included in the study. Whether an intervention study, an observational cohort study or a case-control study, the selection of the subjects can influence both internal validity and external validity; and further, can modify the hypothesis being tested. Internal validity is the quality controlling whether a valid assessment of cause and effect can be made within the context of the study. External validity relates to the generalizability or application of this cause and effect assessment to other populations, and is clearly a secondary issue; if the study has very low internal validity, the conclusions are likely to be wrong, and so its generalizability is irrelevant. With high internal validity, the valid assessment of the causal relationship may be widely generalizable, and does not require that the participants be representative of those to whom the new evidence will be applied. The value of good studies is in the fact that their results can be applied to very different populations, particularly in the future. Thus to choose the best treatments, physicians apply the results from internally valid studies, usually randomized trials, often done in different countries on patients diagnosed many years previously. We do not need to assume that the subjects involved in these earlier studies are representive, in a general way, of the new patient. Similarly we apply knowledge of genetics from fruit flies to humans, because the biological relationships are generalizable although the individuals studied are not. An epidemiological example is the UK Biobank cohort study: whereas its aims are in policy terms (‘improving the prevention, diagnosis and treatment of a wide range of serious and life-threatening illnesses’), these objectives are met through valid comparisons within a large cohort of volunteers, only about 5.5% of those approached, not a ‘representative’ sample of people.1 In contrast, other epidemiological studies may have a different purpose: to measure the frequency of something, and here representativeness is critical. So the appropriate selection of subjects will depend directly on the objective of the study. If we fail to recognize that, we may have problems. Or, we may hope that our study will do everything, and may seek support on the basis both of the assessment of causality and the measurement of health states, as both are relevant to planning interventions. But there are usually trade-offs between the best study designs for these two different objectives. Rothman et al.’s commentary2 mentions the U.S. National Children’s Study, which shows this. The very name of the study implies a nationally applicable study, and in the original design a major strength was ‘the large, nationally representative, equal probability sample design’.3 However, the objectives are stated in terms of causal relationships, to ‘examine the effects of the environment … on the growth, development, and health of children across the United States’, and do not mention representativeness.4 The planners have modified the sampling design, because to achieve the internal validity necessary, an equal probability sampling design would not be feasible. The objectives of assessing causality vs describing characteristics of US children are addressed in a background document.5 This describes the conflict between arguments for probability-based sampling, emphasizing external validity, and for other approaches ‘led by, but not limited to, epidemiologists’, emphasizing internal validity. However, in this commentary the alternatives to survey type probability-based sampling are described mainly as convenience sampling and volunteer sampling, and the concept of external validity given confuses statistical inference with scientific inference, an error that Rothman et al. highlight. Strong epidemiological designs are more complex. The participants, defined by exposure or outcome, are selected to give valid inferences for the eligible groups they represent; but they do not need to be ‘representative’ of a source population in a general way. In another cohort study, with prenatal recruitment of children in New Zealand, the options of representativeness and internal validity are perhaps better balanced; although probability-based sampling was rejected by both logistic and ethical limitations, a multimethod approach in one region of the country has been chosen primarily to achieve good internal validity, with reasonable external validity.6 Both in the planning of studies and in their assessment, a clear understanding of the objectives and therefore the key issues of study design is vital. Rothman et al.’s contribution2 gives a valuable viewpoint on this. Conflict of interest: None declared.