Researching research

In this issue of the journal, El-Boghdadly et al. present their analysis of the academic productivity of research grants awarded by the National Institute of Academic Anaesthesia (NIAA) between 2008 and 2016 1. This is the first time that such an analysis has been presented in the anaesthetic literature and provides food for thought as well as feedback for further strategy for the NIAA. Judged against the intentions of the Royal College of Anaesthetists' Academic Strategy Report 2, we think it is fair to say that the NIAA is a success. As intended, it has served the function, both to the profession and to the world at large, of emphasising that scholarly endeavour is important. At a time when the proportion of anaesthesia-related publications of globally published health research is falling 3, attempts to promote meaningful research in our profession are essential. Funds for research are limited and, consequently, the application processes are very competitive. Those with the responsibility for awarding grants have to balance a number of factors. Should they fund a large number of small grants, or vice versa? Is it appropriate to support young or previously less active researchers, or back the ‘safe horses’ of more established and well published senior investigators? El-Boghdadly et al. report a correlation between the grant applicant's h-index 4) and application success, the number of publications and citations achieved. It is difficult for publication track record not to be a factor as many projects will build upon previous work. Senior investigators have the ‘grantmanship’ experience, which will also give them an edge in this process. However, they may also have greater ‘needs’, in terms of academic staff to employ; indeed their own positions within universities may depend on performance assessments based on grant income. On the other hand, inexperienced researchers without such ‘commercial’ pressures might conduct excellent, low-cost ‘garden shed’ projects with great clinical impact 5. There is also an argument (made by Dick Pels, sociologist of science) for the ‘unhastened’ pursuit of science; that is, where academic enquiry can proceed at a necessarily thoughtful and considered pace 6. This utopian model is clearly in conflict with the pressure to bring in grant income and demonstrate timely outputs and impact, enshrined in various research assessment exercises (see: http://www.ref.ac.uk/). The breadth of our specialty raises dilemmas. Should the NIAA favour one sub-specialty, or try to encourage all of them? Funders may be tempted to support current ‘hot topics’: El-Boghdadly et al. suggest that critical care and peri-operative medicine are awarded more grants but we do not know whether this simply reflects better quality applications in these fields, a preference among the funders, or both. Further compromises must be made between supporting basic science work and clinical applied studies 7. The Australian and New Zealand College of Anaesthetists (ANZCA) has established a Clinical Trials Network to support investigator-led and, what they regard as ‘public good’ trials, where they profess to seek the definitive answers for the best practice that is relevant to practitioners, patients and policy makers [http://www.anzca.edu.au/ctn. Accessed 07/02/2018]. The ANZCA also helps nurture less-experienced investigators such that, rather than independently competing for funding, they can pitch ideas to peers and use their feedback further to develop the concepts to reach the stage of a grant proposal. This initiative has produced many publications with very high clinical impact 8-10. The NIAA, together with the Rosetrees Trust, has established a UK Peri-operative Medicine Clinical Trials Network (POMCTN: www.pomctn.org.uk), which has a stated aim to identify, develop, support and coordinate efficient delivery of large-scale clinical trials in order to improve the health outcomes of patients receiving peri-operative care in the UK. The network also has innovative ‘investigator schemes’ for prospective researchers of varying experience that may well prove to be an excellent programme for nurturing new talent. In 2014, the Lancet published a series of articles about waste in research, identifying problems with all aspects of the research process, from prioritisation, regulation, design, conduct and analysis to reporting 11. The authors suggested that methods of improving the yield from basic research should be investigated; the transparency of processes for prioritising important uncertainties should be increased; new research should always be preceded by a systematic appraisal of existing evidence; and sources of information about research in progress should be enhanced, and used by researchers. A recent follow-up survey of the websites of 11 UK research funding organisations found that only six explicitly stated that they require publication of full reports of the research they have funded 12. An accompanying editorial further discusses this notion of ‘waste’ 13. One criticism of El-Boghdadly et al.'s analysis is that it focuses on publication as the key measure of productivity. The true impact and effectiveness of research is much broader. Archie Cochrane, the epidemiologist after whom the Cochrane Collaboration is named, recounts how, as a Cambridge undergraduate, his professors’ view was that the ‘best research is utterly useless’ 14. Now, there is considerable pressure to deliver patents and other intellectual property, or other measures of public benefit from the research, not confined to ‘publications’. However, Cochrane's advice reminds us that many major research discoveries (like penicillin and X-rays) came about by chance, in the pursuit of something unrelated, often with no tangible benefit apparent at the time. This is analogous to financial investment in well-established companies likely to yield a good overall return as opposed to buying shares in higher risk, fledgling companies with the faint hope of discovering the next Amazon or Apple. Where funding is limited, relying on serendipity alone is unlikely to be a useful strategy and our aim should be to steadily improve the quality and safety of anaesthesia care. Journal publication is only one way of assessing the value of research. Journal impact factors (which reflect how extensively articles are cited 15) are widely traded as the ‘currency’ of science. Indeed, this journal is pleased to have been able to publish NIAA-funded work on chronic pain 16, peri-operative coagulation management 17, and animal or bench type work that may have clinical impact in the future 18, 19. There are, however, other measures of how research is viewed and shared, such as visits to journal and other websites. Altmetric scores (www.altmetric.com) are becoming popular and include, but are not limited to: peer reviews on Faculty of 1000 (https://f1000.com/); citations on Wikipedia and in public policy documents; discussions on research blogs; mainstream media coverage; bookmarks on reference managers like Mendeley (www.mendeley.com/) and mentions on social networks such as Twitter. Wider evidence of impact is sought annually by the UK National Institute for Health Research (NIHR) using the online tool ResearchFish (www.researchfish.net). As well as publications, the NIHR seeks information about: influence on policy, practice, patients and the public (for instance, membership of guidance committees or citation in guidelines or policy documents); patient and public involvement in the research undertaken; intellectual property (assessment tools, new analytical techniques, web applications and patents); and engagement activities such as media interviews, press releases, use of social media, events and workshops. This is requested from researchers for five years after the completion of the work in question, and provides a more extensive picture of the short-term impact of academic endeavour. Of course, the benchmark of research success must be its uptake into practice; however, as El-Boghdadly et al. note, this is hard to quantify and can take many years. Finally, we find it disappointing that 20% – one in five and representing ~£700,000 of funding – of NIAA grant recipients did not respond to the survey. The anaesthesia community has a right to know how these resources are being utilised. The centres concerned are listed in the online appendix to El-Boghdadly et al.'s paper 1. The NIAA has the support and input from many talented researchers and is performing a laudable role in supporting academic anaesthesia. This is now being extended into nurturing and mentoring researchers rather than just ‘funding research’. Such education and ongoing support to grant recipients will not only be particularly useful to those with less experience, but will also be a source of encouragement to more seasoned investigators. It would be interesting to follow the careers of these early stage investigators to assess the impact of this sustenance; a concern of course being if there is any ‘research waste’ 13. Coordinating large multi-centre trials to address important issues in clinical practice, in a manner comparable to the ANZCA Clinical Trials Network, is a commendable development. Perhaps this could be extended to other areas, such as basic science, where there would be potential to share resources and equipment as well as expertise. A strong research culture is very important for the status and development of our specialty and will ultimately benefit our patients. Various specialist societies also provide funding for research under the auspices of the NIAA, but, understandably, this is often limited. As an alternative to these small grants, these societies might be able to work together and identify common themes across their areas of interest, to contribute to larger funding initiatives (which might also be achieved through even higher membership subscriptions). Perhaps the NIAA could also work with major funding bodies such as the Medical Research Council UK or, at least, coordinate and assist with applications to them. It is encouraging to see the foundation for anaesthesia research moving in the right direction with the NIAA playing a crucial role. MI and AS are Editors of Anaesthesia. No other competing interests declared.