Invited Commentary: Using Area-based Socioeconomic Measures—Think Conceptually, Act Cautiously

Investigators dedicated to understanding social inequalities in health in the United States face major challenges. Key among them are data constraints, limitations in theory development, and disciplinary divides. A central data concern is that many data sets do not include adequate health and socioeconomic information. Growing conceptual interest in contextual influences on health leads some investigators to link census to health data in order to use area-based socioeconomic measures (ABSMs) to estimate contextual effects (1). When data permit, both microlevel socioeconomic variables and ABSMs are included in multilevel models to estimate area impacts on health above and beyond the characteristics of individuals (2). However, investigators have also used ABSMs to substitute for missing microlevel information, raising the difficult question of how well these aggregate variables serve to proxy individual characteristics. Investigators of this question have often warned of the dangers of making inferences about individual-level relations from aggregate-level variables, including ABSMs (3–12). It is not unusual for researchers to find substantially larger estimated effects when using aggregate variables compared with when using microlevel ones. Researchers from diverse fields have derived the conditions under which overestimates are likely (9, 11, 13). Overestimates are likely when, had both variables been included in the same regression equation, the estimated coefficient on the aggregate variable would have been larger than—and of the same sign as—the estimated coefficient on the missing microlevel variable. In the case where the explanatory variable is an indicator of socioeconomic position, this condition is met when the aggregate variable represents a broader construct than the microlevel variable or when there are contextual effects (8, 11, 12). Subramanian et al. approach the question of comparability between ABSMs and microlevel socioeconomic variables through an empirical analysis. They offer empirical evidence of contextual effects and argue that ABSMs are not interchangeable with microlevel variables but measure the broader construct of “a shared collective or contextual dimension that all individuals within a context share” (14, p. 829). Thus, their findings suggest that the conditions that other investigators conclude would most often set the stage for overestimating microlevel effects when using ABSMs are met in their analysis. Yet, Subramanian et al. conclude the opposite, stating that their study “demonstrates, at least in the case of disparities in (low) birth weight, that aptly chosen ABSMs provide estimates of health inequalities that are comparable with those based on individual-based socioeconomic measures. The risk, if any, in the absence of individual-based socioeconomic information is a conservative estimate of socioeconomic inequalities in health” (14, p. 833). In light of contradictory conclusions by other investigators, have Subramanian et al. demonstrated their claim, as they say they do? I believe the evidence they provide falls short of making their case. Their confidence in ABSMs is misplaced. Analyzing Massachusetts data for 1990, Subramanian et al. compare estimates of the effect of socioeconomic position on birth weight using three ABSMs, each measured at the census tract or block group level, with microlevel maternal and paternal education variables. They report that estimates from the two approaches are roughly comparable and, in cases where they differ, that the estimate using the ABSM is lower than that using the microlevel variable. However, also studying health outcomes and performing empirical work in the spirit of that reported by Subramanian et al., Geronimus et al. (11) and Geronimus and Bound (12) concluded that ABSMs have significant limitations as proxies for unavailable microlevel variables and are likely to overestimate effects compared with microlevel measures. Subramanian et al. cite those papers (11, 12) as “an earlier assessment questioning the validity of ABSMs” (14, p. 833) but do not engage in a scientific discussion of how to reconcile the dramatic differences in conclusions. Complementing the empirical analyses, Geronimus et al. (11) explicated a statistical model illuminating two potential sources of bias inherent in using an ABSM to measure individual characteristics: a downward bias due to measurement error and an upward bias that arises if the aggregate variable has an independent effect on the outcome. Geronimus et al. formalized this logic algebraically by constructing the hypothetical situation where the coefficient on an aggregate and a microlevel measure would be estimates of the same parameter. From this idealized model, they could illuminate the nature of how the real world falls short. The statistical model of Geronimus et al. (11) led them to predict that most often the aggregation bias would predominate, inflating estimates compared with a microlevel measure. Their empirical work supported this prediction; still, they noted the potential to arrive at underestimates and, in fact, discussed an example where they found this to be true. A critical problem with using ABSMs to impute individual-level effects, they concluded, is that the resulting bias can be substantial and the direction of the bias cannot be interpreted with confidence. One possibility, then, is that the finding by Subramanian et al. represents an instance where an effect is underestimated. Another possibility is that Subramanian et al. arrived at underestimates because of the way they specified the socioeconomic variables being compared. The most common specification used in the literature when substituting aggregate variables for missing microlevel variables is to define the aggregate variable as the location-specific mean or median of the missing microlevel variable and to assign each individual that value (for income, education, and so on). This approach defines the microlevel and aggregate variables to be as comparable as possible. In the hypothetical world where there is no residual confounding, regressions using the microlevel and aggregate variables would estimate the same parameter (refer to Appendix A). Following this approach, Geronimus et al. (11) compared the coefficient estimate for a dummy variable representing whether or not an individual had graduated from high school with the coefficient estimate for an aggregate variable that they specified as the percentage of area residents who were high school graduates. Subramanian et al. specify their socioeconomic variables in a different way. At the microlevel, they categorize mother's or father's education as less than high school, high school, some college, college graduates, and missing education data. They specify their aggregate education variable as areas with these adult population fractions with less than a high school education: <15 percent, 15–24.9 percent, 25–39.9 percent, and 40 percent or more. Setting aside the missing category for parental education, at each level Subramanian et al. allow for six possible contrasts. How does one know which of the six contrasts between the microlevel categories are to be compared with which of the six contrasts between the aggregate categories? Their specification approach provides no intuitive guidance. There are no contrasts where one would expect that, hypothetically, in the absence of residual confounding, the coefficients on the microlevel variable and the ABSM would estimate the same parameter. Instead, their categorization puts circumstances such as being a college graduate on the same plane as being a resident of a census tract with <15 percent high school drop outs. Subramanian et al. highlight contrasts between best and worst. How meaningful, though, is comparing odds ratios for individuals who are college graduates versus individuals who have less than a high school education, on the microlevel, with odds ratios for census tracts with <15 percent adult population with less than a high school education versus census tracts with 40–100 percent adult population with less than a high school education on the aggregate level? Further, theoretically, “best to worst” is the appropriate contrast for some research questions but not for others where comparisons between high school graduates and either college graduates or those with less than a high school education are pertinent. Won't results depend on which contrasts are compared? In fact, the ABSM that best approximated the microlevel maternal education estimate of “best to worst” (while being slightly lower) is the fraction less than high school aggregated to the census tract level. The conclusions of Subramanian et al. draw heavily on this comparison. However, using the same ABSM when the comparison is between college and high school graduates yields an estimated birth weight difference that is now 25 percent higher than that yielded by the maternal education variable. In addition, Subramanian et al. would have arrived at qualitatively different findings had they used a more standard specification. I used the numbers Subramanian et al. reported to calculate rough approximations to the estimates they would likely have obtained had they specified the ABSM fraction high school graduates as a continuous variable and the microlevel variable as a dummy variable indicating high school graduate status. This specification takes the common approach of defining the ABSM as a location-specific mean or median and approximates that used by Geronimus et al. (11), thus facilitating a comparison between the two sets of findings. Using this specification, I estimated that, far from yielding a conservative estimate, the ABSM yielded a coefficient roughly three times the magnitude of the microlevel variable (refer to Appendix B). At a minimum, these calculations suggest that the size and direction of difference between estimates based on aggregate compared with microlevel socioeconomic measures are highly sensitive to specification, a shaky basis on which to form the conclusion that one empirical comparison alone “demonstrates” the comparability of such estimates or the direction of any bias. Moreover, without clarifying the importance of their specification to their findings, Subramanian et al. imply that investigators who specify their aggregate variables in more standard ways should interpret their estimates as conservative, when, in fact, their estimates will more likely be inflated relative to those using microlevel socioeconomic variables. Subramanian et al. qualify their conclusion that ABSMs well approximate microlevel variables as pertaining to the use of “aptly chosen” ABSMs. By “apt” they refer to 1) ABSMs that they would classify as “measures of deprivation” rather than “measures of affluence” and 2) ABSMs aggregated to the census tract rather than to the block group. These are post hoc conclusions and open to question. The view by Subramanian et al. that ABSMs can be classified as “measures of deprivation” or “measures of affluence” is a means of reconciling their finding that results using two of the ABSMs (fraction of the adult population with less than a high school education; fraction of the population living below the poverty level) better approximate results using microlevel variables than when they use their third ABSM, the fraction of the adult population that is college educated. At both the census tract and block group levels and, when comparing best and worst categories, each of the first two ABSMs yielded results that closely approximate those based on microlevel education variables. Increases in birth weight going from the most disadvantaged to the most advantaged socioeconomic levels are 61–79 g for the microlevel variables and 55–72 g for these ABSMs. In contrast, the results using the third ABSM suggest a smaller increase, only about 44 g, associated with going from areas with the lowest fraction of college educated to those with the highest. The authors interpret away the weaker results for the third ABSM by classifying it as a measure of “affluence,” while classifying the ABSMs yielding estimates consistent with their conclusion as measures of “deprivation.” They then argue that ABSMs reflecting deprivation are better for imputing microlevel effects than are ABSMs reflecting affluence. By implication, the results from using the ABSM referring to the fraction college educated can be disregarded. Subramanian et al. do not even present results for this variable when they report estimated odds ratios of low birth weight by socioeconomic position. Results for the fraction college educated are weaker than the others when comparing best and worst categories, but not as clearly so in other comparisons, and, in any case, the explanation Subramanian et al. offer is implausible. The correlation coefficient between the fraction with a college education and the fraction with less than a high school education is 0.73, suggesting that these two ABSMs are unlikely to tap very different constructs. Moreover, whether or not the fraction college educated leads to weaker or stronger results depends on which contrast is studied. When the contrast is between mothers with less than a high school education and those who are high school graduates, it is the college ABSM that yields estimates that most closely approximate the results from maternal education. Calling some ABSMs measures of deprivation and others measures of affluence overinterprets the meaning of an ABSM as reflecting “deprivation” or “advantage” based on its name alone (12). Instead, on average, one would expect that advantaged areas would be characterized by a high fraction of the adult population being college educated, a low fraction with less than a high school education, and a low fraction in poverty, while disadvantaged areas would be characterized by a low fraction of the adult population being college educated, a high fraction with less than a high school education, and a high fraction in poverty. Subramanian et al. also conclude that census tract level ABSMs are preferable to those aggregated to the block group level. This argument is based largely on their interpretation of the second set of empirical analyses that they report, which they refer to as “mutually adjusted” models. These consist of birth weight equations that again contain a few covariates but this time include ABSMs and microlevel education variables in the same models. More often called “multilevel models,” the rationale is to estimate whether contextual variables have an independent association with health outcomes after controlling for individual characteristics. The literature estimating just such models is large, and much of this published work (1, 2, 15) reports results similar to the general findings of Subramanian et al., who identify their analysis as novel, because it represents “the first time that ABSMs measured at multiple levels of geographies have been considered in a simultaneous multilevel manner, thus allowing an interpretation of the relative importance of different levels” (14, p. 829). Subramanian et al. conclude that census tract-level ABSMs were “considerably stronger predictors of birth weight than were block group ABSMs” (14, p. 829), because 1) more often than not, coefficient estimates are higher for census tracts than block groups when both are in the same equation (although the differences are rarely large and the authors do not test the statistical significance of the difference), and 2) “the size of the census tract-based socioeconomic differential was similar to that of individual socioeconomic position from models that do not mutually adjust for each other” (14, p. 829). However, the latter rationale is also true of block group ABSMs for the contrast on which Subramanian et al. focus and even more so in other contrasts. Moreover, when comparison is made of coefficients in the “mutually adjusted models” with those in the earlier models where only an ABSM (at the census tract or block group level) or the microlevel education variables are included, the microlevel coefficients evidence stability, while the ABSM coefficients are reduced substantially (on the order of 60–80 percent). The block group-level ABSM coefficients appear to be reduced more than the census tract-level coefficients are, suggesting that the block group-level ABSMs are more highly correlated with the microlevel variables. The comparisons outlined above suggest the following provisional advice to investigators hoping to use the findings of Subramanian et al. to aid them in making the most apt choice of ABSM level: Block group-level ABSMs are preferable for imputing individual characteristics, while census tract-level ABSMs are preferable for measuring contextual effects. However, the differences between using census tract and block group ABSMs are small. This is consistent with earlier findings comparing ZIP Code with census tract-level ABSMs (12) and should provide some comfort for investigators who have only census tract-level data to them ZIP Code for that Further, the interpretation that the census tract level is the better choice when estimating multilevel models would be if it were based on to the rather than only on the of in an Subramanian et al. the use of ABSMs in a of where their value be Subramanian et al. appear to the of microlevel variables and those of ABSMs. that shared or contextual can health does not imply that ABSMs are the more appropriate way to measure that than to ABSMs as a social in the face of data is better by a of an empirical analysis is to and then how best to use data to do investigators who the of influences on health that are through between by the social of find microlevel variables to test the that their to they find it appropriate to use ABSMs as However, they would have no to to measure through an ABSM rather than at the individual level, even when they interpret as a social Subramanian et al. are to their that assessment well be the only means to the and nature of birth weight at as the individual-level socioeconomic data are unlikely to be level for and less than 25 (14, p. 829). The is than percent of college suggesting that most their by their those who to the than percent of mothers are in for those who in or to school, it is how Subramanian et al. that the birth weight of a to a will be if its higher levels of education at a A other to using an ABSM when a microlevel variable is the case of on one could use microlevel variables. if one is about the that a has not had the to one can microlevel information or use information on the or the of to measure socioeconomic microlevel variables. In contrast, if one is in the to which to outcomes associated with to maternal one to for socioeconomic characteristics Subramanian et al. find that ABSMs very estimates for those mothers who are most likely to have Subramanian et al. do not their that ABSMs can be for that their study findings The basis for this conclusion is their finding that, among to mothers their census tract poverty ABSM socioeconomic in birth weight that were lower but similar those by the microlevel parental education variables. However, using microlevel variables as the standard of comparison is an way to their case. the argument in of using ABSMs is that microlevel measures unlikely to be (14, p. 829), then that ABSMs as well as microlevel measures is an many of their and statistical analyses have for Subramanian et al. do not engage the of ABSMs to questions in and health as the for using ABSMs. However, is made to health by how closely using ABSMs using microlevel variables in health Their empirical are for the value of using ABSMs to provide of the health of residents of are However, they can also when used to the question of whether are better or (14, p. 833). if an area in one census is to have a in the the possibility that this or social of the population is one if residents at the of the were by or or and, were not in the In such the health of the population could well have the By so the possibility of Subramanian et al. missing population including that are in some In it is to expect health to be by characteristics of both individuals and the in which they or with which they are However, between the impacts of individual and characteristics is and because of the of the social as well as for the that individuals are to of to social From a measurement the that microlevel variables only individual characteristics is as is the that aggregate variables only contextual constructs. the coefficient on each will from the other Yet, as as health data sets offer few other social will to use ABSMs to proxy for missing microlevel variables or to estimate contextual effects. They should not do so use of data with due to the limitations is different from the use of data with When using ABSMs, investigators should in study be about the and assessment of these limitations with of theory and that are to the the question of the comparability of ABSMs to microlevel measures is not for on how well ABSMs in the empirical evidence to the dangers of ABSMs as if they were microlevel variables. to understanding of social in health will be to the that investigators draw on a of and an approach to measurement discussion about the of conceptual and of the social of health for research measurement and the interpretation of empirical results in light of measurement for ABSMs also from the for better in the and of one could estimating by a regression of on However, if one had data only on the means of then one could estimating by a regression of on both and are independent of then either will estimate In such case, it would be to use an aggregate variable as a proxy for a microlevel However, the that is independent of is unlikely to in In that represents only a of socioeconomic position and that, among other other of socioeconomic position. In this case, would expect that and estimates of will to the effect of alone on the variable. circumstances will this to be more true when use rather than The use of will the larger magnitude coefficient when both are included in the same the coefficients on the two variables are of the same sign In the context where is an indicator of socioeconomic position, this will be true if either the aggregate variable represents a broader construct than the microlevel variable or the aggregate variable the contextual effects (8, 11, 12). At the aggregate level, Subramanian et al. estimate that from census tracts with than percent population with less than a high school education to tracts with 15–24.9 percent with less than a high school education birth by roughly Thus, an approximate percentage in the with less than a high school education is associated with an difference in birth from a census tract with less than percent to one with between 25 percent and percent with less than a high school education birth weight by g, or roughly g percentage from a census tract with less than percent to one with 40 percent or more with less than a high school education birth weight by g, or roughly g percentage These numbers suggest that, had Subramanian et al. used the ABSM fraction in the census tract with less than a high school education and measured it as a continuous they would have estimated a coefficient between and At the microlevel, the that roughly the same of mothers each of the categories high school some college, and college the contrast between those with and without a high school should be roughly the of the contrast between those with less than a high school education and each of these three about Thus, had Subramanian et al. specified their microlevel and aggregate socioeconomic variables to the way that Geronimus et al. (11) they would have found that the ABSM fraction in the census tract with less than a high school education yielded a coefficient roughly three times the magnitude of the microlevel variable. The is to and for on a of