Measuring production efficiency of acute hospitals in Scotland, 1991-94: validity issues in data envelopment analysis

Data Envelopment Analysis is a potentially useful technique for measuring production efficiency and is being increasingly applied to health care services, as with other areas. However, some fundamental concerns need to be addressed before Data Envelopment Analysis can be accepted widely as a routine tool. These include issues of validity and specification error. This paper suggests a broad framework for assessing validity, which should be developed further, and gives illustrative examples of the problems in a real application within health care. Data from 75 Scottish acute hospitals for three years from 1991-94 are analysed. Internal validity is assessed by examining the effect on the results obtained of the choice of inputs and outputs selected for analysis. External validity is assessed by examining differences in the results obtained from the data for each of the three years. In both cases, it is concluded that there is a disturbing lack of concordance between the different results obtained. Caution must therefore be exercised in using Data EnvelopmentAnalysis in the health sector and in looking at its results, particularly those from a single time period.