Toward Better Descriptions of Innovations

Innovation is important. In academic medicine, new and better ideas are key drivers of progress in the discovery of new cures for disease, in the care of patients, and in the education of tomorrow's physicians and scientists. But what are the best ways for articles in Academic Medicine to share information about innovations? This question leads to even more questions, both general and specific. Is it better to publish brief descriptions of a large number of promising ideas or, instead, in-depth case studies of major ideas? What are the most important questions to ask about an innovation? Must an author demonstrate the generalizability of an innovation? Is it necessary to have evaluation data that show that patients are satisfied or that students have learned something or that residents' or physicians' performance in a particular area has improved? I feel a pressing need to address the issue of describing innovations because, among the manuscripts I have decided not to publish over the last several months, a number are reports of potentially important innovative ideas, projects, and programs. One of the main reasons for not publishing such a report was that it was often a description of a project followed by an evaluation that assessed only participants' satisfaction, knowledge acquisition, or both. The problem is that we already know that it is possible for expert academic physicians and scientists to implement a novel project or procedure that leads to participants' satisfaction and/or learning. These manuscripts — which basically answer the questions “What was done?” and “Did it work?” — are at best reporting something we already expect (i.e., an innovation led to improved participants' satisfaction and/or learning) and are, at worst, missing an opportunity to share with readers important aspects of innovations that could improve teaching, research, and/or patient care. So how do we reach beyond “Did it work?” to a more useful approach to describe an innovation? An innovation can be a creative approach to a known problem (e.g., a new and better way to assess the performance of residents or medical students on standardized patients; a new approach to organizing the care of patients in an academic setting; a substantially improved measure of faculty productivity; or a novel way to facilitate discovery by restructuring basic science departments on the basis of emerging themes). (For the sake of completeness, I should mention that an innovation can also be a novel creation altogether — e.g., a completely new style of music or painting. However, this type of innovation is much less common in medicine or science, where almost any creation can be conceptualized as a response to a need.) If an innovation is a creative solution to a problem, it is important to address questions relevant to problem-solving.1 I propose that what would be most valuable to those who seek creative solutions to problems is not just reading another person's report of the results of implementing an innovation, but in addition, reading a reflective, analytical, and scholarly treatment2 that, to the extent possible and appropriate, satisfies the following criteria: There is a clear and thorough description of the problem. Why is the problem important and relevant? How is the problem situated within a more general context of education, research, or patient care? How is it situated within the environment of a medical school or teaching hospital? How is it situated within a broader conceptual or theoretical framework? How and why was the problem identified? What motivated the recognition of a problem? What were the preexisting conditions that enabled the problem to occur or persist (e.g., financial or political issues, institutional values, traditions)? How did individuals' values or assumptions change to allow the problem to be recognized? Why was the problem not recognized previously? Was the problem defined by the needs of a group of people or by a deficiency in an organization or by some other difficulty? How does the problem affect the organization? There is a statement about the degree to which the problem is generalizable. Does the same problem exist in other venues? Do different stakeholders view the problem differently? Key issues of the stakeholders are stated. Who is affected by the problem and how? Who are the decision makers? What are the objectives of the decision makers? There is a delineation of the array of potential solutions. What is the full range of solutions available to address the problem? What solutions were available in the local environment? What solutions might be available to others in other environments? Is there a theoretical basis for all or some of the solutions? The details of why a particular solution was selected and/or developed are presented. Is the chosen solution innovative (i.e., new and better)? How is it qualitatively different from prior approaches by other individuals? Why is it a reasonable idea? How is the solution situated in the larger context of education, research, and/or patient care? What strategy was used to select this solution? What are the assumptions underlying the selection of this solution? Why were other solutions rejected? What generalizable and local factors affected the choice of this particular solution? What co-existing factors made the selection of this particular solution possible or desirable? Is there evidence or analysis to support the choice of this solution and to discard other potential solutions? The implementation of a particular innovative solution is described. How was the solution implemented? What principles, concepts, or theories guided the implementation? What barriers were encountered? What did and did not work? What lessons were learned along the way? What institutional factors allowed the innovation to work or impeded its implementation? Did the implementation of this solution change the participants in some way? Did it change the environment in which it was implemented? What was the timeline for implementation? What was the cost? What milestones were used as indicators of progress? What metrics were used to evaluate the solution and what did they show? Was the solution implemented successfully? What was learned as a result of implementation? There is a critical analysis of the quality of the innovative solution. What have others done that is similar, related, or analogous? What distinguishes this innovation from similar efforts? What value does it add? Are there other perspectives that affect the problem or solution (e.g., political; legislative)? What unforeseen events affected the quality of the solution and its implementation? What are the preconditions needed to implement the innovation in another setting? There is an assessment of the innovation's potential influence on the field, discipline, or area of study. How could the knowledge gained as a result of implementing this innovative solution change thinking or practice in education, research, or patient care? What assumptions existed in previous work that were uncovered or debunked by implementation of the current innovative solution? Has the innovation propelled a project, program, or discipline forward or in a new direction? Could it do so eventually? Could the innovation change the definition of a discipline? Could the innovation cause a program or discipline to be seen from a new point of view? Does the innovation represent an incremental step forward (i.e., enabling something to be done more efficiently) or could it eventually alter practice in a substantive way?1 There is an account of the degree to which the innovation described is a sustained innovation. Did the innovation generate new ideas or other innovations? Did it identify new problems? Did it open new avenues for further exploration?3 The approach embodied in these nine criteria is based on the assumption that it is often the problem itself and the choices made along the way that are most interesting and most generalizable to other contexts, and not a particular solution that works best in a certain, local setting for parochial reasons. Also, these criteria highlight that sustained innovation3 is important to success in academic medicine. (Again, for the sake of completeness, if an innovation is a new creation, rather than a response to a problem, deficiency, or need, a report of it would differ somewhat from one that addressed the nine criteria stated above, since there is no problem to describe. I propose that in this case, the criteria above could be modified to include a clear and thorough description of the innovation, including how the innovation is qualitatively different from existing approaches and is situated in the larger context of education, research, and patient care; a list of other innovative ideas that may have been considered and a discussion of why each was rejected; and an account of how the innovation affects stakeholders and institutions, including how different stakeholders may view the innovation differently.) In sum, a primary aim of academic medicine is to find innovative ways to treat disease, to take care of patients, and to train the next generation of physicians and biomedical scientists. Consequently, it is important for the journal Academic Medicine to publish articles about these new and better ideas. A simple, “Did it work?” approach to reporting such innovations is often insufficient (i.e., it addresses only some of the questions in just two of the criteria above, numbers 1 and 6), fails to explore a broad range of key issues, and misses the elements essential to those who work on similar issues (e.g., details about the problem and its generalizability; an account of the full array of solutions considered and why each was accepted or rejected; a critical appraisal of the potential of the innovation to spur additional innovation). Thus, I propose the criteria above to help advance thinking and stimulate discussion about describing innovations. I do not mean to suggest that every manuscript about an innovation should satisfy each and every criterion I have presented above. Nor do I mean to imply that the journal is not interested in high-quality articles that report the results of a rigorous study of an innovation that shows sustained benefit at acceptable cost. Rather, my hope is that the criteria above will catalyze broader and deeper reflection and lead to a more critical analysis of new ideas, projects, and programs. Ideally, a report of an innovation will provide not just information but insight — the kind of insight that will sustain a cycle of progressive thinking that will lead to even newer and better ideas. • The issue of describing innovations was a topic of discussion at the Spring 2008 meeting of Academic Medicine's editorial board. The discussion was energetic, substantive, and charged. I thank the board members and the journal's professional staff for their ideas and suggestions, and for stimulating my thinking on these issues. Individual members of the board and staff may not agree with each point in this editorial. But, as a group, by engaging in that brainstorming activity, we were able to extend our thinking farther than any one of us could have alone. Steven L. Kanter, MD