The right wood, but barking up the wrong tree

Smith and Petticrew's article1 makes some very salient points about the current short-sightedness of public health evaluation, yet I cannot agree with their analysis in its entirety. There are some defects in the authors' own vision, which lead them to bark up the wrong tree. They do make a strong and persuasive argument about the narrowness, and inadequacy, of much of the evaluation in the public health field today. Important macro-interventions to tackle the social determinants of health and inequalities are not only complex in terms of having multiple components and outcomes but also because they often have the properties of complex systems and need to be evaluated as such, following the advice of Alan Shiell et al.2 This includes assessing the full societal impact of the disease or intervention in question, not just health-related effects upon individuals. The examples of transport interventions and urban regeneration given in Boxes in the article are good illustrations of the issue of complex societal impacts that need to be assessed. Their call for greater development of methods of analysing and evaluating complexity is welcome, as is the advocacy of using techniques familiar in other fields in novel ways to address public health questions. ‘Macro-evaluation’ approaches for ‘macro-interventions’ is a point well made. I would, however, take issue with the authors on two main points in their analysis of the source of the problem. First, the ‘blame’ for the observed narrowness of current approaches is put squarely on ‘the public health profession and the disciplines that contribute to it’, who cannot see the wood for the trees, because they have, according to Smith and Petticrew, ‘maintained the historical focus upon detailed micro (individual and health sector) analysis and not broader (societal and system) analysis’ (p. 4). There is certainly plenty of that sort of analysis going on, but equally, there are many researchers within the public health community who take the macro-perspective and attempt to take on board all of the evaluation challenges mentioned in the article. Notable areas in which the wider macro-approach is strong include tobacco control, child poverty and health, health impact assessment of non-healthcare policies and interventions, the impact of politics and of globalisation on health, health systems analysis, to name a few. One of the authors, Mark Petticrew, is himself leading a programme of research on evidence synthesis of non-healthcare policies and their impact on public health, which is exemplary in this respect. Even in the field of communicable disease control, which the article implies is more straightforward, the trend is towards more comprehensive analyses, encompassing, for example, the social and economic impact of the disease in question and the way in which the control system operates to support or constrain behavioural change. This applies to both the big global programmes on malaria, TB and HIV/AIDS as well as the more neglected tropical and other infectious diseases. Such a sweeping criticism of the whole field of academic and professional public health is therefore misplaced: there are plenty of researchers who share their view of what needs to be done and are attempting to work with a macro-perspective. On the contrary, I would contend, it is the funding bodies and commissioners of evaluations who cannot see the wood for the trees and who have had a powerful influence on the types of study that have been supported. You only have to look at the outcome specifications and study design requirements in public health-related research calls (and the successful awards) to see the micro-level approach that the funders are favouring. Second, the authors conclude that ‘Policy too supports this micro-level methodological focus’ (p. 5) and cites the Wanless Report as taking a paradoxical stance. In my view, rather than merely ‘supporting’ the micro-level approach, official policy is leading or prompting the approach. The Wanless Report was a review carried out by a banker (Derek Wanless) of the prospects and directions for public health, commissioned by the UK Treasury, and both the economic slant and the narrow focus on changing individual behaviour were clearly visible in the end product. Around the same time, the Treasury itself conducted a comprehensive spending review of what all government departments (excluding the treasury) were doing to tackle health inequalities, which produced narrow, behaviour-focused, recommendations and resources to address health inequalities. Policy documents, media reports and funding calls from national funding bodies all continue to emphasize micro- over macro-approaches, and NICE continues to give more weight to the types of evidence that the authors are rightly critical of (though to be fair to NICE, it is now attempting to broaden its approach in the public health field). It is these actors who are in most need of having their awareness raised, rather than preaching to the converted. The challenge in the final sentence of the article could more appropriately be directed at them.