Evidence‐based Evaluations of Pace

To the Editor: I join the readers of JAGS in a sense of shared professional pride in the preliminary successes of the PACE replications expressed in the recent article by Catherine Eng and her colleagues.1 The accompanying editorial,2 however, engendered a feeling of chagrin with our apparent lack of research sophistication. The lead sentence captures the essence of the editorial: “One needed only visit and observe the On Lok Program in Chinatown in San Francisco … to be convinced that here was an approach to fully integrated long-term care for very frail persons living in their own homes that really worked and that saved money as well.” The essence of my chagrin is that I think we have to move beyond professional judgment — no matter how finely honed, and Dr. Williams' is second to none — and move to evidence-based evaluations. Succinctly, and in today's jargon, we can expect to be asked to “show me the data.” The growth and development in research sophistication has identifiable steps: (1) a good idea for a better mousetrap is essential (and kudos to the creativity emanating from On Lok and the PACE sites); (2) the development of concept feasibility (the 10 years of concept development at On Lok may seem excessive, but this is a complex concept; the S/HMO may have an equally long concept demonstration phase); (3) the development or evolution of research designs (including Phase I studies in drug research, refinement of measurement techniques and gold standard definitions, documenting the natural histories, etc.); and (4) implementation of research and evaluation designs that unambiguously clarify a cause and effect sequence for treatment effectiveness and cost-efficiency. The last step is the sine qua non for evidence-based policies. Without the last step we are left with simple professional judgment, a circumstance in which individuals of good will can disagree. And what happens when the last step is not taken and professionals disagree? It is not pretty.3,4 We can speculate about what will happen when the disagreements go beyond card-carrying gerontologists and geriatricians (for example, to an HMO policy board meeting that is considering adding home care to its usual treatment options for frail older people). If we are asked for definitive evidence to convince the skeptic of the effectiveness and efficiency of our innovations in long-term care models such as the PACE model, we cannot yet provide it, and we will not be able to provide it without progressing to the next step in our research and evaluation sophistication. We need to move beyond professional judgment to evidence-based paradigms. These are the current rules applied to innovations in clinical interventions, to pharmacologic interventions, and to health promotion/disease prevention interventions in general and breast cancer screening in particular. We will have to learn how to apply the evidence-based rules to our field if we are to be successful in convincing the skeptics. Editors note: The above letter was referred to the author of the original article, and his reply follows.