How Well Can Method Substitute for Data? Five Experiments in Poverty Analysis

No one doubts that good data are essential to sound policymaking. Alas, data are invariably faulty. Methodological solutions to data inadequacies have often been proposed and implemented, but they have been tested only rarely. Yet the methods that are used may well determine the direction of policy. For example, the particular survey method used—and the way nonsurvey data are interpreted—may be critical in assessing whether a country's strategy for reducing poverty is working. This article shows how counterfactual experiments can help test the reliability of various methods of dealing with common data problems. 'Well-designed methods—and they need not be very complicated—can help get around the problem, although it appears that substituting method for data is a long way from being perfect. Objective data obtained from representative surveys of living conditionsare widely used to stimulate public awareness of poverty and motivategovernment actions to benefit the poor. Yet analysts and policymakers routinely find that these data are deficient in one or more important respects and must find credible methods for dealing with those deficiencies. Various so-lutions have been proposed that rely on certain regularities in living conditions and use a relatively small number of more easily measured variables—such as membership in certain predefined socioeconomic groups—to infer the missing data. Hidden differences in living standards may well confound such efforts. Even though the partitions commonly used in assessing poverty—such as land owner-ship or region of residence—reveal large disparities, these disparities may be weak indicators for targeting the poor. In fact, some recent research indicates that variations between socioeconomic groups are often dwarfed by differences