Evaluating TQM: The Case for a Theory Driven Approach

Speaking to General Accounting Office (GAO) evaluators in 1991, Total Quality Management (TQM) guru Joseph Juran lamented how woefully inadequate our understanding was of his progeny's impact. Wrote Juran (1991, p. 51): As far as measuring the [TQM] results that have been achieved, there's a big information vacuum out there. Hardly anything useful is going on as far as evaluating Unfortunately, little has changed in the interim to negate Juran's observation substantially. Indeed, a paucity of systematic and rigorous evaluative efforts still exists, thus prompting accusations that is little more than a religious doctrine that admonishes the unwashed to just do it, in the blind hope that results will follow (Sims et al., 1992, p. 138). Although Juran was largely concerned that flawed or nonexistent evaluations of would limit its application in both the private and public sectors, more pedestrian concerns ought to make systematic and rigorous evaluations of interventions attractive to public managers and scholars serious about improving quality. First, interventions in the public sector can be costly endeavors. For example, in one small city with 700 employees, initial training cost $75,000, with training costs over the ensuing four years running $30,000 annually (Walters, 1992). Second, applications are increasingly viewed with skepticism in the private sector, a skepticism influenced by anecdotal and empirical analyses stressing its decidedly mixed results (Mathews and Katel, 1992; Dumas, 1989; Clemmer, 1991; Mathews, 1993). Despite proponents' warnings not to expect short-term cost savings, many CEOs have grown cynical upon seeing little change in their companies earnings statements. What is more, Mathews (1993, p. H1) reports that private sector consulting firms in the 990s are thriving on the TQM repair work spawned by miscarriages and abortions crafted in the 1980s. Third, the wholesale adoption of unreformed techniques in the public sector has been questioned widely Swiss, 1992; Kronenberg and Loeffler, 1991; Cohen and Brand, 1993; Durant and Wilson, 1993). Programs, for instance, with dense contextual goals (Durant and Wilson, 1993) or with more diffuse decision structures than those typically found in private businesses may inordinately slow the implementation of TQM, thus diminishing the likelihood of success (Juran, 1991). Finally, although advocates typically counsel public agencies to pick the lowest hanging fruit first (i.e., to selectively apply interventions in areas most likely to succeed), they tend to offer little advice as to how public managers might recognize these situations in practice or convincingly monitor their progress and assess their results. Arguably, only rigorous evaluation of interventions in public agencies, as well as the testing of hypotheses culled from these experiences by researchers, can address such issues. How can public managers, program analysts, and scholars best approach evaluations? We argue that evaluations and research must become more theory grounded and contingency based than has historically been the case if practice, cumulative knowledge bases, or theory-building are to improve significantly. Explicitly delineated models of change processes, the assumptions underlying them, and the ways in which variables in the models are expected to interact to produce these outcomes must inform any serious attempt at evaluation or theory building in the public sector. Moreover, lest these models, assumptions, and interaction patterns are eventually specified and progress toward their achievement is explicitly assessed, TQM's half-life in the public sector could be shortened appreciably. More precisely, ways must be found to identify, assess, and demonstrate short-term, nonmonetary progress toward both intermediate and ultimate goals of cultural change and quality improvement. Otherwise, the private sector's emerging cynicism and disenchantment with TQM's inability to show short-term monetary benefits could spread rapidly to the public sector as well. …