The role of simulation modelling in public health policy evaluation

As obesity and diet-related chronic diseases become a public health threat in almost all countries, population-based government policies to improve diet quality have been recognised and increasingly implemented as major public health interventions.1Mozaffarian D Angell SY Lang T Rivera JA Role of government policy in nutrition—barriers to and opportunities for healthier eating.BMJ. 2018; 361k2426Google Scholar When evaluating public health interventions, various methodological approaches—ranging from implementation science to impact evaluation to comparative effectiveness research—could be applied to provide evidence on which interventions work best for whom and under what circumstances.2Spiegelman D Evaluating public health interventions: 1. Examples, definitions, and a personal note.Am J Public Health. 2016; 106: 70-73Crossref PubMed Scopus (20) Google Scholar Evidence acquired from conventional study designs is sometimes insufficient to inform policy decisions. They tend to focus on short-term health outcomes (eg, changes in weight or biomarkers), might not represent a diverse population, and are unlikely to evaluate all policy-relevant options. Simulation modelling can help fill this evidence gap by informing multiple relevant processes, testing a multitude of plausible scenarios that would be impractical and infeasible in trials, quantifying the magnitude of intended and unintended consequences, and having the option to adjust and refine designs before a trial or actual implementation testing in the real world.3Kim DD Neumann PJ Comparative modeling to inform health policy decisions: a step forward.Ann Intern Med. 2019; 171: 851-852Crossref PubMed Scopus (5) Google Scholar In The Lancet Public Health, Zoé Colombet and colleagues4Colombet Z Robinson E Kypridemos C Jones A O'Flaherty M Effect of calorie labelling in the out-of-home food sector on adult obesity prevalence, cardiovascular mortality, and social inequalities in England: a modelling study.Lancet Public Health. 2024; 9: e178-e185Summary Full Text Full Text PDF Scopus (2) Google Scholar used policy simulation modelling to predict the long-term health effects among the adult population in England of implementing the menu calorie labelling policy, which mandates large food businesses to display product energy information for individual items. Conceptually, individual consumers might consume less food or items with lower energy density in response to the calorie information displayed, and the food industry might also reformulate food products sold in large food businesses to contain fewer calories. Existing evidence supports both pathways to some extent, but there is insufficient evidence on how the potential reduction in the overall calorie intake would result in decreased bodyweight and BMI, which in turn reduce the risk of dying from cardiovascular diseases, among many possible outcomes. The modelling study estimated that the current menu calorie labelling policy would reduce obesity prevalence by 0·31 percentage points (95% uncertainty interval 0·10–0·35; note that 25·9% of adults in England were classified as obese in 2021 to 20225UK Office for Health Improvement however, the estimated impact on obesity prevalence is similar to a 2013 modelling study of a 20% tax on sugar-sweetened drinks in the UK, which estimated a step-wise reduction of 1·3% (95% uncertainty interval 0·8–1·7) in the number of adults with obesity6Briggs AD Mytton OT Kehlbacher A Tiffin R Rayner M Scarborough P Overall and income specific effect on prevalence of overweight and obesity of 20% sugar sweetened drink tax in UK: econometric and comparative risk assessment modelling study.BMJ. 2013; 347f6189Crossref PubMed Scopus (187) Google Scholar (the 0·31 percentage point reduction in obesity reported by Colombet and colleagues would be equivalent to around a 1·2% reduction, given a prevalence of 25·9%). Considering the small effect size, Colombet and colleagues suggest that expanding the menu calorie labelling policy to all out-of-home food businesses would have much larger effects (2·65 percentage point reduction in obesity prevalence and 9200 cardiovascular disease deaths prevented). Along with Colombet and colleagues' important contributions and thoughtful analyses, a few additional points need to be considered to better inform policy decisions. First, the study only measured the health impact regarding reductions in obesity prevalence and cardiovascular disease deaths. The absence of other health outcome measures (eg, diabetes, cardiovascular disease cases, and quality-adjusted life-years) would limit the comprehensive assessment of the comparative advantage of the calorie labelling policy over other public health interventions. Second, the analysis mainly focuses on projecting health benefits but does not consider economic consequences. It would be imperative to capture economic effects (eg, implementation costs and potential downstream cost-offsets in health-related expenditures) to determine whether additional economic resources are worth the additional health improvement.7Kim DD Basu A How does cost-effectiveness analysis inform health care decisions?.AMA J Ethics. 2021; 23: E639-E647Crossref PubMed Google Scholar Similar US-based studies included formal cost-effectiveness analyses that accounted for the effects on multiple diet-related health and economic outcomes,8Liu J Mozaffarian D Sy S et al.Health and economic impacts of the national menu calorie labeling law in the United States: a microsimulation study.Circ Cardiovasc Qual Outcomes. 2020; 13e006313Crossref Scopus (18) Google Scholar, 9Dupuis R Block JP Barrett JL et al.Cost effectiveness of calorie labeling at large fast-food chains across the U.S.Am J Prev Med. 2024; 66: 128-137Summary Full Text Full Text PDF PubMed Scopus (3) Google Scholar which is a research gap in the UK. Finally, the study showed that the effects of the menu calorie labelling policy were similar in the most and least deprived socioeconomic groups, based on the current evidence of equivalent policy effects across socioeconomic groups. As cardiometabolic risk factors are highly correlated and are more prevalent among individuals with low socioeconomic status than in those with high socioeconomic status,10Schultz WM Kelli HM Lisko JC et al.Socioeconomic status and cardiovascular outcomes: challenges and interventions.Circulation. 2018; 137: 2166-2178Crossref PubMed Scopus (682) Google Scholar Colombet and colleagues' other assumption (ie, that the effect of BMI on cardiovascular disease mortality only varies across age groups) means they are likely to underestimate the potential benefits of BMI reduction on cardiovascular disease mortality among the most deprived socioeconomic group. The distributional analyses across socioeconomic groups would have been more precise if the simulation model fully incorporated subgroup-specific effect sizes and surrounding uncertainty. Despite the known challenges in assessing the long-term effects of public health policy interventions, predictions are often necessary to make an informed, evidence-based decision. When carefully developed and validated policy simulation models are used alongside robust sensitivity and scenario analyses, simulation modelling provides a framework for integrating multiple data sources, reflecting the strength and uncertainty of evidence, incorporating heterogeneous effects across population subgroups, and ultimately identifying cost-effective strategies for improving population health and health equity. I declare no competing interests. I acknowledge funding from the National Institutes of Health National Heart, Lung, and Blood Institute (R01HL115189). Effect of calorie labelling in the out-of-home food sector on adult obesity prevalence, cardiovascular mortality, and social inequalities in England: a modelling studyThis study offers the first modelled estimation of the impact of the menu calorie labelling regulation on the adult population in England, although we did not include a cost-effectiveness analysis. Calorie labelling might result in a reduction in obesity prevalence and cardiovascular disease mortality without widening health inequalities. However, our results emphasise the need for the government to be more ambitious by applying this policy to all out-of-home food businesses to maximise impact. Full-Text PDF Open Access