Methodological progress note: How to handle race in research according to the research aim

The National Institutes of Health (NIH) Revitalization Act of 1993 requires the inclusion of minorities in federally funded clinical research, mandating that all research proposals describe the anticipated participants' racial diversity, and address the impact of race and sex on health and disease. Although the NIH makes up a small share of clinical research, this initiative has pressed the discussion on how to best handle race in research design, collection, and analyses. To avoid worsening disparities and perpetuating negative access to healthcare, the use of race data in research studies requires careful consideration. In this issue of the Journal of Hospital Medicine, Yaeger et al. highlight problems that may arise when including race as a covariate in analyses, noting "the decision to include or exclude race in analyses is challenging."1 Before considering covariates in statistical model building, the purpose of the research goal as explanatory or predictive should be made clear. The distinction between these two different research aims affects the analytic strategy for handling race in the analysis (Table 1). Association aim—assess burden of disease or differences in the incidence of the outcome by race. Causal aim—avoid race and consider more direct measures of racism, health care access, socioeconomic status, or chronic adversities. Potentially include race if there is evidence of improvement in model calibration because of differences in the incidence of the outcome by race. Evaluate the performance of the model by race. Evaluate model fairness by race. Explanatory research identifies factors that are causally related to, or associated with, an outcome. Explanatory model building aims to identify individual factors (e.g., risk factors) associated with the outcome and mitigate confounding. Confounding variables, those related to both the risk factor and outcome, may create spurious associations between the risk factor and outcome if not accounted for in the analysis. Under the goal of explanatory modeling, individual regression coefficients and p values are of key interest. Addressing confounding either in the study design or in the analysis plan is critical in interpreting the relationships of interest. Predictive research aims to identify the combination of factors that best predict whether a disease or condition is present (diagnostic) or best predict whether future clinical outcomes will occur (prognostic). Predictive modeling emphasizes overall predictive accuracy; the role of individual variables is less important. Variables may be included in the final model even if they are not causally related to the outcome as long as they are good predictors of that outcome. In studying racial disparities, including race in the analysis to determine associations with the burden of disease or differences in outcome incidence may be of interest to shed insight on existing disparities without implying a biological cause. The underlying mechanisms driving race associations are likely to be complex and multifactorial. Race should rarely be included in models with a causal aim between race and outcome. For causal inference, as Yaeger and colleagues suggest, the social construct of race could be decomposed into causal elements such as more direct measures of racism, access to health care, socioeconomic status, or the presence of chronic adversities.1 The investigator should carefully consider whether race should be included as a confounder as the more direct causal elements underlying race associations may better address confounding. Similar to the explanatory model setting, there has been substantial debate on whether to include race in predictive models. The argument to include race in clinical prediction models is motivated by the potential to improve predictive accuracy when diagnostic or prognostic differences exist between racial groups, recognizing that the mechanisms for these differences are likely complex, multifactorial, and not biologic. The argument to exclude race in clinical prediction models includes a history of misuse and unintentional outcomes of racial profiling or fear of widening racial disparities. When reporting prediction model results, authors should follow the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines.2 If race is included in the prediction model, authors should provide clear justification for doing so because of the potential misuse and unintentional consequences that may result. For example, in developing the stroke risk prediction tool, Hong et al. reported "given known differences in stroke risk by race, the race variable was included as a key exposure, understood as a social rather than biological construct" and evaluated the model by race and found "all algorithms exhibited worse discrimination in Black individuals than in White individuals, indicating the need to expand the pool of risk factors and improve modeling techniques to address observed racial disparities and improve model performance."3 To avoid increasing health disparities, Paulus and Kent propose a practical framework for the evaluation of clinical prediction models.4 They introduce the concept of algorithm bias and fairness. Bias refers to model attributes that may result in differential model performance in different racial groups. Fairness refers to the model predictions that may create discriminatory or unjust impacts in different racial groups. Although the group of interest here is race, these concepts are also applicable to other underrepresented groups like sex and ethnicity. Paulus and Kent propose two steps to reduce the bias that may arise through differential model performance across racial groups: (1) use representative samples for model development (e.g., racially diverse samples); and (2) assess subgroup validity by evaluating the performance of the model (e.g., discrimination and calibration) overall and in each racial group. Fairness concerns are specific for models with a clear beneficial decision outcome such as models used to allocate available donor organs. Fairness may be of concern even when prediction is not biased. Paulus and Kent propose two fundamentally different unfairness mitigation approaches: (1) input-focused to promote race-unaware allocation by strictly avoiding the inclusion of race or race proxies in the model and (2) output-focused approach to evaluate model output fairness (e.g., equal error rates) and addressing fairness violations through the use of fairness constraints which systematically reclassify patients to equalize allocation between groups or by applying different decision thresholds across groups. Models should include only well-established, causal risk factors. Models should be race-unaware and exclude noncausal variables that may be race proxies (e.g., zip code). Evaluate model fairness (assess the difference in model error rates between groups, such as false negative rate between the groups). Ensure fair distribution of beneficial outcomes by either using different decision thresholds across race groups or by applying fairness constraints to strategically reallocate the beneficial outcome based on race. For further reading and guidance on including race in prediction models, the prediction model risk of bias assessment (PROBAST) tool was developed to assess the risk of bias and methods of clinical prediction models.5 The Quality Assessment of Prognostic Accuracy Studies (QUAPAS) tool is a similar risk-of-bias tool specific for prognostic tests, including those of single biomarkers, multimarker scores, and imaging studies.6 Both of these resources can be useful in planning the prediction model approach in addition to assessing bias in the developed model. The authors declare no conflict of interest.