The story of GBD 2010: a “super-human” effort

What has working on the international, multi-investigator Global Burden of Disease 2010 been like? Pamela Das and Udani Samarasekera asked the researchers involved. Massive, collaborative, and challenging are just some of the words that the scientists who worked on the Global Burden of Disease Study (GBD) 2010 have used to describe the project. It is not difficult to see why. GBD 2010, involving nearly 500 researchers from more than 300 institutions in 50 countries, has not only been a large data-driven research endeavour but also a huge exercise in diplomacy, management, and leadership. The results are set to shake up health priority setting in countries and the world, providing an unbridled amount of up-to-date, comparable data on the diseases, risk factors, disabilities, and injuries facing populations. How did it all start? In 1990, physician and health economist Christopher Murray at Harvard University and medical demographer Alan Lopez at WHO embarked on the first ever attempt to measure the global burden of disease and developed the now-famous Disability Adjusted Life Year (DALY) metric that made it possible to combine estimates of mortality and morbidity burden around the world. They, along with a dedicated team of junior analysts at Harvard, who, says Lopez, “worked day and night, with comparable desperation to us, and for little money”, produced the first GBD. Since then, there have been estimates in 1999–2002 and 2004. But the latest iteration of the project, GBD 2010, has been a different beast altogether, involving a vastly greater number of collaborators, countries, and computers, as well as an expanded objective. 20 years ago, the project assessed the burden of 107 diseases and injuries and ten selected risk factors for the world and eight major regions over one calendar year. Now, thanks to advances in technology, the availability of data, and the participation of experts around the world, as well as the leadership of a core group of GBD researchers, the scope has swelled to 291 diseases and injuries in 21 regions, for 20 age groups, and an estimation of trends from 1990 to 2010. GBD 2010 also includes an assessment of 67 risk factors. Rafael Lozano, who was part of the core GBD team led by Lopez and Murray at the Institute for Health Metrics and Evaluation (IHME), University of Washington, WA, USA, provides a sense of the enormity of the latest undertaking: “One example of the size of the project can be represented with the cause of death database that we have used for this study. We spent almost 5 years building it; we have included almost 800 million deaths from 1950 to 2010, and the data come from different sources. The goal was to incorporate ‘all the available data’. I am sure that we are still missing a lot, but in my knowledge, this is the biggest database for cause of death analysis in the world.” “Running the programs to map the data to our cause list of 291 causes and correcting the bias can take days, even using a powerful cluster of more than 100 computers. The data that we have to store after the modelling process can take 3 terabytes.” Lozano estimates that the storage needed for the causes of death data was 400 times bigger than that for GBD 1990. But the study challenged man as much as it did machine. When The Lancet asked several researchers what it was like to work on GBD 2010, most described it as “exciting”, “demanding”, and even “frustrating”. Lopez, who is now an affiliate professor at IHME, and head of the School of Population Health, University of Queensland, Australia, says that GBD is “a way of life, rather than another massive research project”. 2 years ago, the study was way off schedule, there was a huge data lacunae, a need to revise analyses so that the same or comparable methods were used throughout, and to tackle the diseases, injuries, or risks where little work had been done in the first 3 years of the project. Murray, who is the director of IHME, described the completion of the effort as “super-human”. “It has literally occupied the vast majority of every day for me over the last 2 year sprint to the finish line. This intensity of effort on the part of many of the leaders of the study and a large number of other researchers over a prolonged period was exhausting. But in many ways it was also the best part; it has led to some great work and stimulated lots of innovation from across a broad group of researchers and created a genuine sense of camaraderie among those involved. The worst part has been dealing with various political aspects that are inevitable when trying to quantify in a comparable way many different problems. The results matter, so people and institutions get very engaged”, he tells The Lancet. Abraham Flaxman, another core team member at IHME, was trained as a mathematician/computer scientist, and working on GBD was his first introduction to global health. He called it a “trial by fire”. For Theo Vos from the School of Population Health, University of Queensland, the lead author of the paper on years lived with disability, working with so many disparate expert groups stood out. However, “it felt sometimes like I was herding cats”, he quips. For those leading the disease expert groups, the experience was also rewarding. George Mensah, who led the cardiovascular group, says, “It provided a unique opportunity for me to use many skills and experiences acquired at the US Centers for Disease Control and Prevention and in academic cardiology in the preceding 20 years”. Wagner Mercenes who led the oral conditions expert group says he “learnt a great deal about coordinating a large project and developed the ability to accommodate the views of a large number of experts in different diseases”. Majid Ezzati, chair in global and environmental health at the School of Public Health, Imperial College London, UK, and part of the GBD core team leading the comparative risk assessment, tells The Lancet, “Intense is probably the best single word to describe the science, atmosphere, and perhaps even interactions; as you can imagine, depending on the day that could be very good or exciting, or not so good!” Lopez says that the organisational challenge of collecting, analysing, and feeding back findings to collaborators to get further inputs into the estimation process could sometimes require several rounds until scientific consensus was reached; “and this for about 300 diseases, over 1100 sequelae of these diseases, and almost 70 risk factors! The sheer scope and magnitude of the study was a major challenge in itself”. Furthermore, Lopez adds: “We needed to convince scientists who were experts in particular diseases, injuries, and risk factors to share expertise and data with us in order that our estimates might be based on the best available evidence. That has been demanding and moderately successful.” For Harvey Whiteford, professor of psychiatry and population health at the University of Queensland, assembling the global epidemiology for mental disorders had specific problems. “Mental disorder, as an illness concept, is understood differently in different cultures and this impacts on how it is identified in epidemiological surveys. There is an under-reporting of mental disorder in some cultures due to stigma (which as always led to an underestimation of the burden of those disorders). There are significant limitations in data availability from many parts of the world and inconsistencies in the ways data have been collected.” He explains that although the group's initial systematic searches identified almost 100 000 data sources, fewer than 700 of these could be used after the team applied their inclusion criteria. Furthermore, he says, “The variability in study methods made comparability a challenge. For example, estimates from smaller community-based surveys of mental disorders were often the only data available for low-to-middle income countries. Rather than discard these data, we used the statistical platforms specially developed for GBD 2010 to utilise as much of the data as possible. However, it was certainly a challenge to undertake this work at the same time as the modelling tool, DisMod-MR, was being refined.” Ezzati says that making decisions on these data issues pushed the team both scientifically and intellectually: “As researchers, we tend to believe that more ‘data’ are better than less. I still believe so. But more data, but not all the perfect data we could wish for, means that we need to fundamentally think differently about when to stop searching for more and how to use it. This is partly an analytical issue but partly one of scientific judgment and intuition: how do you ‘feel’ that more data hunting will not turn up more or will not fundamentally change the conclusions.” But technical issues were only part of the problem. As Kenji Shibuya, member of the GBD core team and chair of the Department of Global Health Policy at the University of Tokyo Graduate School of Medicine, Japan, says: “If an issue was purely technical, it would be resolved by discussion—but the numbers could become very political and clashes of strong egos would take place—that would require fine diplomacy…[and] academia is generally not good at that.” Ezzati also notes that diplomacy was a key component of the project, especially in view of the different types of researchers involved. “The study brought out the well-known but far too frequently overlooked issue that people of different scientific traditions—clinicians, basic scientists, epidemiologists, and quantitative scientists—think and speak differently about the same problem. This can of course be a very powerful resource for bringing together different ways of looking at a problem and solving in the most comprehensive and interesting way—a true systems approach. It can also be a challenge, and at times a source of tension. Again, managing it takes patience, effort, and the ability to be able to step back and not only see, but also truly and deeply appreciate, the other perspectives and their contributions.” The intellectual and organisational challenges came at a price—because GBD 2010 took 2 years longer than planned, funding sources ran out while much work remained to be done. As Catherine Michaud, a core team member and senior research scientist at the School of Public Health, Harvard University, notes, the limited available funding “posed inherent challenges” itself “and required creativity on the part of key collaborators to raise additional funding as well as true commitment to contribute countless unpaid hours to the project”. The GBD has brought new dimensions and advancements to both health measurement and the way global health is viewed. “Better methods to better extract truth from poor quality or missing data has fostered tremendous scientific innovation”, says Lopez. “At the same time, the poor quality of data encouraged us to derive ways of estimating uncertainty around all GBD outputs and findings. This has been a major scientific achievement, but it will also greatly assist with the interpretation of our findings for policy.” He adds that “the GBD study has effectively provided a global health data ‘audit’ whereby there is now a massive repository of national health information, but also substantially more knowledge about its utility and reliability for informing health debates”. For Felix Masiye, head of the Department of Economics, University of Zambia, and coordinating member of the GBD consultation meeting in Zambia, “the idea to achieve international comparability that employs a metric that encompasses a widely shared notion of health is a unique contribution of the GBD”. Also unique to GBD is its comparative risk assessment approach, which tries to ensure that the methods used to estimate the burdens attributable to major potentially modifiable risk factors such as high blood pressure, diet, tobacco smoking, and ambient and household air pollution, are consistent among the risk factors. “This allows their respective burdens to be compared with regard to their importance globally and among global regions”, says Aaron Cohen, co-chair of the GBD 2010 expert group on ambient air pollution, and principal scientist at the Health Effects Institute, MA, USA. “There have been many assessments of the health impacts of ambient air pollution, and the number of such assessments is growing, but only GBD places its estimate in the context of other major risk factors with which policy makers and the public must contend”, he adds. Shibuya points out that “GBD has also shown to the world that new ideas and methods outside the conventional medicine and public health framework—economics, philosophy, psychology, political science, environmental science, computer science, engineering, etc—play a significant role in the estimation and interpretation of the disease burden”. An important advance has been the improvement of methodological approaches. Vos admits, “we now joke to each other how we used to get away with murder in the past doing burden [studies]. We tended to make lots of not so replicable ad-hoc decisions and few people knew what we were doing anyway. This exercise has deliberately tried to minimise the ad-hoc decisions and instead aimed to maximise what information we can get from the data. We also have much more closely involved hundreds of experts and young researchers who are very capable of looking over our shoulders and are much more articulate at picking apart the estimates. This is mostly to the good (and has led to many improvements in the methods and results)…although we have also had some experiences of disease experts taking on roles of advocates with aim to ‘boost the numbers’ rather than being ‘impassionate’ scientific advisers”. From an academic standpoint, Ezzati would like to see a broader contribution: “that ‘descriptive epidemiology’ of the sort that GBD 2010 is, but was also done by the likes of [Richard] Doll and [Richard] Peto or Sam Preston, becomes a more central part of research and training—epidemiology as describing the health of individuals and populations including, but not solely, eliciting the causes of differences in levels and Murray now it is the to large analytical to not on the same the GBD is an example of applied to global do the work an amount of data, huge statistical analytical and the expert of many from around the world. The GBD that this is possible to However, while the of the study on is and the researchers on GBD 2010 the the health of become a GBD for example, have the burden of in and countries, leading to and in the disease and in Lopez says: “If we could see the GBD 2010 findings being widely used to health policy priority setting in and to raise about the need to health information systems in poor countries, that would be a massive But with different and estimates of global burdens of disease available in the public what countries do when trying to work out their national health Lopez that the estimates and he do not think that we be trying to and certainly not to scientific and about how best burden of disease might be done. this be for countries, it not be as or the quality and limited availability of data on which the estimates are often But countries to the of different and interpretation of WHO has estimates of disease burden not make He says that countries would from the GBD methods to the that is available to on the health of their but which the GBD researchers did not have “This would the reliability of our estimates in some countries, perhaps not in In doing countries might also think of estimating disease burden for of for health “The GBD data is a great for about population He countries to for in the GBD data, this the estimates based on the methods and most comprehensive Furthermore, he notes that GBD 2010 results to the GBD and to national data by might important challenges to conventional This is and policy makers would be to more burden of their who at the of Health and was a central member of the GBD consultation meeting in says the information from GBD 2010 will not only be for countries that do not have good health information systems but also for those with strong a this of exercise comparability and the use of of health. For those countries that do not have information such an exercise will their health conditions and the For such as which have a information and a strong public health academic these are a contribution to the knowledge about the national health the diseases, injuries, and their risk factors in the of conditions of health and public to what the Murray that GBD is to become an even more exercise in the years to “As more data are made available the of global health epidemiology is much more challenging than when there was little data This but when there are the is to a than when there are or in the data that must be and in the In other in global health measurement the of global health more and time less. We need to this challenge into to and the Murray it will be to the GBD on a as to health information that will the GBD take another “The good is that the GBD 2010 methods and a so that will be says with the completion and of the and there is little about what can be done in the the the project like at the that it could be but GBD has always its which is It has also felt the tremendous effort for the researchers involved. Shibuya “As is often the with working with it is like a are so many and and and but most working on GBD you that you are part of truly in the of global This has been The first at on This has been The first at on the GBD risk be used to Global Burden of Disease Study 2010 estimates important new estimates of health burden attributable to 291 diseases and and estimate the health burden with 67 risk factors, into a of as to in global burden and risk factors from in not only the burden and risk for 2010, but also estimates for 1990. for better to the of the new Global Burden of Disease Study The the first GBD in 1990, and to make use of this contribution to global health. its the GBD 2010 a in global health knowledge and our for It will set the of health and funding for years to GBD and of the Global Burden of Disease Study 2010 is a for this we for health. The of scientists from institutions in 50 countries has produced an important contribution to our of and health for countries and the global to 100 000 The Lancet, Christopher Murray and the findings of their 2010 Global Burden of Disease in which that injuries the global population some 300 million years of every of years were the number one of young people and for nearly a of the world of million in 2010, up from million in 1990. of the were and many had that on is not and has been building around the that an is Public is by about the of that can new and expanded of to of the most estimates of and and of mortality by together with data from the Global Burden of Disease Study 2010 in The make it that is not From new estimates to better Global Burden of Disease Study 2010 in The Lancet an effort to global and estimates of levels and trends in the burden of assessment of the and health and trends is for making for public health. WHO GBD 2010, which was by the Institute for Health Metrics and Evaluation with its and on the of many scientists, those who work in WHO GBD a multi-investigator for global comparative data, and findings of the Global Burden of Disease Study 2010 are described in in The Lancet. This large is an of a of work that with GBD The number of diseases, injuries, and risk factors and the of analysis have greatly expanded in the past 20 and change over time has been GBD 2010 the basic of GBD trying to use all the and and health with comparable and epidemiological assessment from or of GBD and Global Burden of and (GBD) is a scientific effort to quantify the comparative magnitude of health due to diseases, injuries, and risk factors by and for specific points in The GBD of the burden of disease is health not income or For researchers, and the GBD provides an opportunity to see the to diseases, injuries, and risk factors, and to in a time, and group what are the most important to health Department of Samarasekera The of GBD a Lancet this Aaron have also been as principal scientist at the Health Effects Institute, MA, USA. This has been made to the as of