Research note: Estimating maternal deaths averted: A field‐based methodology

Tracking progress in the reduction of maternal deaths in developing countries is of great interest at both the national and international levels (e.g., as one of the Millennium Development Goals). To date, there have been two approaches to tracking progress: by measuring or estimating maternal mortality ratios 1 (MMRs, an “impact” indicator); and by measuring activities designed to avert maternal deaths (“process” indicators), such as proportion of births attended by a skilled person, and the availability and use of emergency obstetric care (EmOC) 2, 3. Both of these approaches have their shortcomings. Maternal mortality ratios generally cannot be used to measure change over short periods of time (e.g., 5 years), and wide confidence intervals may make it impossible to say whether a change has actually occurred 1. Moreover, it is difficult to use MMR data to target activities, or to link changes in MMR to program activities. As for process indicators, while they are helpful in targeting interventions, and can show change in a short period of time, they do not satisfy the desire to demonstrate progress in terms of reductions in deaths. In this Research Note, we use data on the utilization and quality of EmOC to estimate deaths averted. These data were gathered using a set of indicators issued by UNICEF, WHO and UNFPA in 1997 that are often referred to as the UN Process Indicators 4. These indicators are now being used in many countries, and are increasingly being incorporated into national health management information systems 3. Table 1 lists the data used to estimate maternal deaths averted, and shows data from needs assessments done in Nicaragua and Chad 5, 6. To estimate maternal deaths averted using the UN Process Indicators on EmOC, we focus on the met need for EmOC, and the case fatality rate (CFR) in EmOC health facilities (see Table 1). Met need is calculated using actual service data on women with serious, direct obstetric complications who are treated in health facilities (Table 1, row d). In the Chad and Nicaragua studies, the facilities included were public and private facilities that qualified as EmOC facilities because the full range of “signal functions” were being performed. The number of women treated is the numerator. The denominator is the estimated number of women in the area who need EmOC (row c). Since the proportion of women who develop serious complications in a population is rarely ascertainable, obstetric complications are estimated as 15% of births in the UN Process Indicators (row c). The number of births expected is, in turn, calculated by using the estimated population in the study area (row a) multiplied by the crude birth rate (not shown). Estimates of the population and the birth rate are usually available from the United Nations or the national government. Met need is shown in row (e). The EmOC Facility CFR is the proportion of women with serious obstetric calculations treated in EmOC facilities who die. The numerator is the number of women with major obstetric complications who are reported to have died in the facilities where they were treated (row f), while the denominator is the number of women treated in the facilities for these complications (row d). Thus, using the UN Process Indicators, we know the number of women treated for obstetric complications in facilities (row d), and how many of them died (row f). To convert these service data into an estimate of deaths averted, what is needed is an estimate of the likelihood that the women would have died had they not entered an EmOC facility. We will call this new statistic the “CFR outside EmOC facilities.” We calculate the CFR outside EmOC facilities by first deriving the total number of direct obstetric deaths outside the health facilities by applying 80% of the MMR (row h) to the expected births (row b) (estimates of national MMRs in 2000 (row h) are taken from WHO/UNICEF publications). Next we subtract from the total direct obstetric deaths (row i) the deaths that occurred in facilities (row f), which gives us the deaths that occurred outside the facilities (row j). This is the numerator. To derive the denominator, we subtract the number of women with complications who were treated in facilities (row d) from the total expected complications (row c). Now we have the ingredients for the CFR outside EmOC facilities: the number of maternal deaths outside the facilities (row j), divided by the number of women with complications who are outside the EmOC facilities (row k). This method of estimating the CFR outside EmOC facilities (row l) recognizes that CFR will vary from place to place depending on services that are not captured in a needs assessment for EmOC (for example, private doctors' offices). Generally, we expect a country with greater resources to have a lower CFR outside of EmOC facilities than a country with fewer resources. As expected, Table 1 shows Chad, a least developed country, has a higher CFR outside of EmOC facilities than does Nicaragua, a country with a stronger economy and health system. As EmOC services expand in a country, and as the quality in these facilities improves, more lives will be saved. An increase in met need for EmOC means that more women with obstetric complications are being treated in EmOC facilities, and a decrease in the Facility CFR means that the proportion of these treated women who die has declined. Thus, the pool of women in the unmet need category is shrinking, and the CFR outside EmOC facilities is applied to an increasingly smaller number of women. Using the UN Process Indicators has been found to be useful for promoting focused action to reduce maternal deaths in a variety of countries. The great benefit of this methodology is that it directly links progress in programs to progress in reducing the number of maternal deaths in a short period of time. Existing methods for measuring maternal mortality ratios do not make this link between action and MMR reduction. There are several potential weaknesses in this new methodology. The most likely error is that the number of deaths being averted is undercounted, for a variety of reasons. If data from needs assessments do not include all facilities providing life-saving services, the met need will be artificially lowered, and this will lead to underestimation of deaths averted. For this reason, we recommend that sampling of facilities during needs assessments be as inclusive as possible, e.g., both public and private facilities be included. In addition, recent experience has shown that facilities that do not provide all the signal functions of EmOC (and, therefore, do not qualify as EmOC facilities) often provide some of the signal functions, and thereby probably save some lives. Where data are available for all facilities (EmOC facilities and other facilities), these data can be used. This would give a fuller picture of all EmOC care provided. Nonetheless, some sources of life-saving care are still likely to be left out, especially care given in smaller clinics and the private offices of doctors and midwives. Another possible source of error is the mis-recording of major direct obstetric complications—which may lead to either over- or undercounting of complications and estimation of deaths averted. Certainly, an important part of monitoring progress is improving records in health facilities, where there is often not even a column in the delivery register for obstetric complications. As for estimating expected complications in the population, there has been concern about using 15% of births as an estimate of the need for EmOC. However, studies from settings as diverse as rural India and the United States have confirmed this as a reasonable estimate 7, 8. Since the MMR estimate is a key ingredient of these calculations, it should ideally correspond to the study area—e.g., a national MMR for a national study of EmOC. If, however, the EmOC needs assessment was carried out in the most disadvantaged part of the country, the national MMR will be too low. It should be noted that data gathered using the UN Process Indicators do not capture progress towards reducing deaths from indirect obstetric complications, which lead to at least 20% of maternal deaths. On the other hand, the signal functions of EmOC do not directly address deaths from malaria, tuberculosis, or HIV, which are leading causes of indirect obstetric deaths. Programs that aim to measure reduction in deaths from indirect causes will need to develop additional indicators. We look forward to a period of learning, as colleagues around the world explore these issues.