Editorial: The Measurement of Progress—Some Achievements and Challenges

Economists working on national income have long cautioned that gross domestic product and other money indicators of economic activity are not good measures of human welfare. National income is usually measured in three equivalent ways based on total consumer expenditure, the value of aggregate company sales or the costs of inputs, but there is a growing global consensus among experts and political leaders for the need to broaden concepts and measurements so that they include the wellbeing outcomes that we hope increased economic activity will deliver (Giovannini et al., 2009). The UK’s current work to reinforce its measures of human wellbeing, led by the Office for National Statistics, is among the latest in a series of related national and international sea change initiatives which make it timely to take stock. Here, we do so by looking briefly at the capabilities approach as the intellectual framework at the centre of recent international developments before identifying some themes and challenges in this area of statistical practice. Equating national income with progress ignores both inequality in the distribution of households’ income and other factors that are relevant to wellbeing such as equality of opportunity, human rights, autonomy, freedom and self-determination, which are increasingly important from a policy and societal perspective to overcome such limitations. Sen (1985) proposed an alternative approach to welfare economics that identifies three core aspects of life quality: functionings, i.e. the vector of things that a person does or is, depends on access to resources; happiness, which depends on functionings; capabilities, which are defined as the set of all functionings that a person could choose, given their resources. According to this approach, we can think of welfare outcomes or benefits to humankind of progress in three different ways: the activities and roles that people engage in; how satisfied they are with those activities; the opportunities and constraints that people face. This suggests a very different way of measuring progress and wellbeing, compared with just looking at changes in gross domestic product, according to which we might look at three kinds of factors or indicators: functionings, life evaluations and feelings, and the capability set. Income still plays a key role within this broader framework, but as an index of resource entitlement rather than an end in itself. In addition, the approach emphasizes the fact that what a person does or is depends on the opportunities and constraints that he or she faces and that people are heterogeneous in terms of their abilities to turn resources into functionings. Capabilities researchers note widespread support for equality of opportunity in many aspects of work and public life. Furthermore, because people can make psychological or behavioural adjustments to unacceptable inequalities, they tend to argue for the importance of measuring the real opportunities that people face, as well as what they do or feel. Closely related, researchers in this area have tended to emphasize the intrinsic value of human welfare and the importance of developing objective measures. The approach was most influential first in development circles where it gave birth in 1990 to the United Nations’s human development index (which measures progress in terms of income, life expectancy and literacy) and subsequently helped to inspire the development of the Organisation for Economic Co-operation and Development’s ‘Measuring progress’ programme, launched in 2004, and the closely related European Union’s ‘http://www.beyond-gdp.eu’ set of activities established 3 years later. In these policy applications, there has been an emphasis on the development of objective, multi-dimensional indicator sets and there is significant commonality between the dimensions that have been proposed (compare, for example, Stiglitz et al. (2009) and the Franco-German Ministerial Council (2010)). The capabilities approach has argued for a multi-dimensional analysis to quality of life and this seems to be widely accepted now even if some issues remain to be resolved (see, for instance, Alkire and Sarwar (2009)). As a result, it is possible to use the approach to map and classify wellbeing indicators according to whether they relate to single or multiple dimensions, as well as to whether they pertain primarily to functionings, capabilities or happiness. If, for the present purposes, single dimensions are grouped into three broad categories (health, economic and social) a typology of indicators emerges as in Table 1. The examples used to populate the matrix cells will be familiar to many and taken together they suggest the viability of developing a balanced portfolio of progress indicators across the wellbeing space. In addition, the taxonomy and examples refute early concerns about the impossibility of measuring potential concepts such as ‘opportunity’ which can be done either directly by developing new data (Anand et al., 2009), or by applying econometric techniques to draw inferences (e.g. Bourguignon et al. (2007) and Krishnakumar and Ballon (2007)) from existing data sets. Furthermore, there is a debate in the literature about whether underlying multivariate data are more or less important than the construction of an index and, in a nutshell, we believe that both are valuable though like Revallion (2010) we believe that underlying dimensional information often does the real work whereas index measures can play more to communication and media needs. For example single-dimension information is of value to government departments for targeting effort or indicating performance and is essential in the estimation of causal models. The Franco-German Ministerial Council (2010) seems to take a similar view by advocating what it calls its ‘dashboard’ approach to its quality-of-life indicator set though it does not make much of the distinction between functionings and capabilities, perhaps because empirical measures can often be interpreted as indicators of either. Types and examples of progress indicators based on wellbeing Types and examples of progress indicators based on wellbeing The final column of our typology acknowledges the presence and use of summary measures which range from multi-dimensional poverty indices, through time use data (as used in Bhutan) to measures of capability (e.g. Anand et al. (2009)). (An alternative summary approach is to ask people about their overall life satisfaction, a point to which we shall return below.) Compared with the technical developments that have been applied to gross domestic product over the decades, development of these aggregate measures has only recently started to move away from the foothills. What we do know is that empirical work in this area often faces a choice at the moment, between using household-type survey data which benefit from a panel structure or large objective data sets which are cross-sectional. We are not aware of any data set in the world that supports the development of aggregate wellbeing measures covering a wide range of dimensions using panel data for more than one country though such a data set would evidently be extremely valuable. In addition, when constructing an index, methods of weighting need to be developed, chosen or compared and in their absence the result is a vector description. That itself may be useful but the most viable weighting systems used in the field seem to be preference weights based on happiness equations or equal weighting based on insufficient reason possibly combined with expert judgement. Clearly indexers will benefit from more research on approaches to the empirical development of weights. Statisticians have long been concerned about issues of communication and the UK’s government public consultations illustrate the increasingly important role that public deliberation (Roberts, 2004) plays in public policy. In a predecessor exercise by another government department, the Department for Environment, Food and Rural Affairs, it was found that members of the population wanted quality-of-life indicators to be broadened beyond environmental issues so it seems reasonable to suppose that the current exercise will provide good information on the range of issues that matter to people. Of course it is perfectly possible to collect quantitative data on quality-of-life perceptions through surveys as a team led by the London School of Economics and Political Science have argued for: Dolan et al. (2011). We believe that data on happiness with life or subdomains could be valuable in policy although we recognize that resourcing decisions tend to be based on ‘objective’ data. Nevertheless, if it were possible to identify subgroups with particularly low levels of life satisfaction, they could be candidates for policy targetting. Furthermore, it would certainly be of interest to know what functioning and capability dimensions contribute most or least to overall life satisfaction—some work on this already exists (e.g. Anand et al. (2011)) but more would be valuable. And finally, we note that international comparisons of happiness are probably best made with care as it is well known that response varies across cultures. Kapteyn et al. (2008) addressed this by using anchoring vignettes to create, in effect, an international scale, against which national distributions of happiness can be compared, so the problem may not be insurmountable if international bodies seek to go down this road. So subjectivity may be unavoidable but equally there are techniques for dealing with some of the issues that will arise (international income comparisons are not dissimilar where for some purposes purchasing-power-adjusted figures are preferred). Attempts to measure progress exemplify how statistics can provide both a scientific description of economies and societies as they evolve and, at the same time, a focal point for public debate about the normative values that are accorded to different dimensions of growth. Indeed, it could be claimed that measures of progress and wellbeing are the ultimate performance indicators and that many of the Royal Statistical Society’s insights into their development and use (see Bird et al. (2005)) carry over naturally into this field. However, it is worth stressing that the capabilities approach has been successful in part because it provides a technically coherent theory which draws on conventional economic analysis but is also conceptually attractive by its engagement with modern concerns about fairness and equality of opportunity. We conclude by looking forward to a couple of challenges with the aid of a prediction and a suggestion. The prediction is that the next major development in this field will focus on sets of progress indicators that are sensitive to wellbeing variations over the life course. Work by Heckman and colleagues (e.g. Herman and Masterov (2004) and Cunha et al. (2010)) on young children has shown how the capabilities approach can support analysis of dynamic issues and goes on to identify the value of parental investments in skill acquisition—the Organisation for Economic Co-operation and Development (2009) has recently completed some work in the area and see also Chevalier (2004). The drivers of wellbeing vary tremendously over the life course and moving beyond a money metric of progress will force us to understand these factors more directly. Secondly, we encourage policy makers to draw on data and research related to wellbeing in their evaluations of policy options. It has been claimed that we should not rush into policy based on wellbeing research but, as Google Scholar lists work on capabilities and happiness with citation counts in the tens of thousands, we argue that there is already a volume of relevant research and that the real challenge is in developing mechanisms to facilitate exchange between researchers and policy makers.