Information for decision making: lot quality assurance sampling in the spotlight

Lot quality assurance sampling (LQAS) has been discussed in international community health since the late 1980s.1 Based on industry, a manufacturer may wish to assure the quality of batches of a product as acceptable or unacceptable. A sample from a particular batch is taken, and the lot is accepted if the sample achieves a predetermined level of quality. If it does not, the entire lot is rejected. This interesting approach has been adapted for use in a wide range of health service contexts in developing countries. In these circumstances, routine health surveillance may be limited and health service information systems sometimes perform poorly. LQAS has been used to assess immunization coverage,2 assess communities in relation to the prevalence of active trachoma,3 identify strategies for yaws control,4 detect malaria epidemics5 and, as reported in this issue, be potentially useful for malaria programme management.6 LQAS appears sensitive to detecting poorly performing parts of the health systems being sampled, and the aim is for this to help local managers take remedial action and locally appropriate decisions. While the small sample sizes required are attractive, the trade off is that some lots will be rejected even if they are in reality acceptable (false positives), while others will contain a higher than acceptable level of failures, but escape detection (false negatives).7 With the transfer from factory to a tool for health services management in developing countries, the innovators responsible have developed manuals and guides to assist in implementation. It is here where Rhoda et al.8 in this issue of the journal point out there are some potentially serious issues. Somehow in the translation to these easy to use tools, they claim, the principles of the original methodology have slipped away. With the null hypothesis, the starting point is, for example, that immunization coverage is low, and LQAS aims to disprove this (and thus show it is above the predetermined threshold). However, in the widely used materials, the null hypothesis has been lost or reversed, starting with the premise that health service performance is adequate. Then, as a result of small sample sizes, type 2 errors mean that poorly performing ‘lots’ will sometimes be missed. Rhoda et al. are also concerned that in the need to translate fairly complex statistical concepts into manuals for health managers without any reference to statistical concepts, something gets lost in translation, and health workers may be misled into a false sense of security. Pagano and Valadez9 provide a robust response, not surprisingly, and reiterate the principles, and refute that hypothesis testing is central to the premise of LQAS, but that LQAS should be seen as a classification tool, rather than a hypothesis test. They also refute a number of other assertions made by Rhoda et al. and refer to the need for field-tested suggestions for improvement. How much is there in the literature evaluating LQAS? There are papers concerning its feasibility;10 others use nested designs from surveys to explore LQAS against conventional sampling,11 showing it to be economical. One paper examined LQAS against reliable independent health systems data, such as Matlab,12 and more studies of this kind are needed. Any debate can help science move forward, and monitoring and evaluation of health services in developing countries is important and could potentially have a large impact. It is clear that this approach to informed decision making is attractive and enthusiastically promoted, and there are many who use it. Certainly, simple systems to detect poorly performing parts within a health system can help to inform management, and that is what is needed; however, it must also be ensured that the recipe books and the users do not inadvertently lose sight of limitations. Indeed, understanding type 2 errors is not difficult for many health staff. The IJE has published much about LQAS over the years and, as with any methodological approach, there is a need for it to be kept under review, with open scientific debates in areas of uncertainty, as we exemplify in the current issue. Conflict of interest: P.G. is the specialist editor and subsequent to his initial involvement with this article Dr Valadez was appointed to his institution as a Professor. From that point on all correspondence and management of these two articles was done without dialogue with Professor Valadez and under direction of the Co-Editor.