What research studies do practitioners actually find useful?

As a relatively recent paradigm, evidence-based information practice requires development at all stages of its cycle. Work to date has targeted such aspects as the questions practitioners consider important, the characteristics of the published evidence base and the use of tailored appraisal checklists. Perhaps, as a consequence, the application and use of research in health information practice has received correspondingly little attention. In previous columns we have attempted to ‘second guess’ by identifying examples of published research that practitioners might find useful, focusing on its results and implications for practice. This begs the question: What research studies do practitioners actually find useful? In Chapter 10 of our recent edited volume on Evidence-based Practice for Information Professionals our Canadian colleagues, Denize Koufogiannakis and Ellen Crumley, look at Applying Evidence to your Everyday Practice.1 They identify three categories of research evidence; evidence that is directly applicable, evidence that needs to be locally validated, and evidence that simply improves our understanding. In our online FOLIO course on Evidence-based Librarianship (April–May 2003) we extended the thinking of Koufogiannakis and Crumley to encompass four categories, defined as: Direct application of results—where the setting in which the research has been conducted is not significantly different from that in which you work. Results of research can be applied with little adaptation. Conditional application of results—where research results look promising but require further exploration (audit, survey, pilot) at a local level before applying the results. Derivation—where some aspect of the methodology or perhaps the instrument (questionnaire or interview schedule) can be adapted to your own practice, though different. Enlightenment—where the research has no direct application to your practice but enhances your appreciation of a particular issue or phenomenon. An example of our additional third category, derivation, is where we took a methodology that used a focus group for evaluating a website (described by librarians at Texas A & M University)2 and adapted it for evaluating a local clinical effectiveness gateway. As part of our course, we asked practising healthcare librarians to identify research which they considered to fall into each category. After briefly discussing the implications of this particular approach, this column highlights examples of each category in turn before focusing specifically on category 1—that is, research which participants considered to be directly applicable to their practice. The perceived applicability of research is very subjective, being determined by such factors as the setting and nature of the reader's current service, external environmental factors, planned future developments and whether the hosting body is a learning organization. Additional moderating variables relate to the personal characteristics of individuals themselves—do they possess strategies for continual updating from the published literature?, do they embrace change or do they have to have it thrust upon them?, do they possess the degree of ‘lateral thinking’ required to apply research from an apparently unrelated area to their own practice? This subjectivity is reflected in the fact that the same article could, and indeed did, appear in each of the four categories depending on the viewpoint of a particular course participant. This phenomenon could be described in terms of a ‘signal to noise ratio’ whereby the strength of the originating message, determined by the closeness of a piece of research to one's own situation, becomes progressively more diluted by intervening environmental and personal variables. So, for example, one librarian may have an updating strategy that limits what is read to only that which is directly applicable, filtering out messages intended for other, more subordinate, purposes. Much research that appears in the health information literature, and by far the largest proportion of research published in the library and information literature in general, falls into the category of research for general enlightenment. Examples in this category from the health information literature include general health service librarians reading about methods of conducting systematic reviews or reading about clinical librarianship or personal digital assistants at a point of time when they have no specific intention to introduce such approaches into current practice. Similarly, reading a study of administrative issues, staff management or leadership in a setting that is different from one's own may alert one to generic issues without necessarily having an immediate application. Such research forms part of a general librarian ‘toolbox’ upon which a professional may draw when attempting to address a newly identified problem or unfamiliar situation. Research identified in this way, and stored in the librarian's consciousness on a ‘just in case’ basis, provides a better understanding of issues involved so that should such an issue arise in the future the librarian already understands the key issues. The potential usefulness of such research is encapsulated in the comments of one Australian librarian: ‘You just never know what will happen next!’ In an evidence-based culture, the derivation of tools, instruments and methods from existing published work must necessarily supplant the endless development of one-off questionnaires of dubious validity. So, for example, one Australian librarian described using a case study at St George's Library in the UK as both background reading for a user survey and as a source of adapted questions. Another librarian identified the potential usefulness of the methodology used for a qualitative investigation of consumer use of the Internet, involving focus groups, usability tests and in-depth interviews, to investigation of more locally specific information needs. This particular application of research allows the librarian to investigate diverse features of their service (e.g. information needs or information use) without necessarily having to develop methods completely from scratch. This preserves their ability to adapt to the changing needs of their service and its surrounding environment. Here, the goal is not simply to prevent ‘reinvention of the wheel’ but, as Muir Gray has more appositely phrased it, to ‘stop us from reinventing the flat tyre’! While strenuously resisting attempts to perpetuate a ‘not invented here’ mentality, whereby research is only utilized when conducted in an identical environment to the context in which it is to be used, we should acknowledge that evidence-based practice requires judicious application of findings from an external source. So, for example, in applying lessons from an information literacy programme for staff nurses to other groups you would want to have a clear picture of the specific characteristics of the group being studied and of any significant differences from groups in your own setting. This may or may not involve replication of the methods used in your local context. Similarly, an initiative aimed at improving outreach to community organizations on the other side of the Atlantic would probably need further local investigation before implementation. In many cases, findings concerning the use of new technologies by a general population, such as found in the Journal of Information Science or Internet Reference Services Quarterly, need to be challenged as to whether they apply equally to specific groups such as health consumers or even healthcare professionals. The main consideration in using this category of research is of heterogeneity versus homogeneity, i.e. is the population being studied more different than similar or more similar than different compared with the one to which I hope to apply the findings? While this could take on the dimensions of a ‘half-full or half-empty’ debate we have found it pragmatic to weight the ‘burden of proof’ by asking a reader to state explicitly reasons why a particular piece of research could not be applied in a local setting. Of premium value is research that is considered of value and is directly applicable (i.e. without too strenuous attempts at translation) to contemporary health information practice. A total of 87 different identifiable research studies falling within this category were cited by 216 participants on the international FOLIO course. The 10 studies receiving three or more mentions are identified in Table 1. Seven of these were from either the Bulletin of the Medical Library Association (BMLA) or its successor, the Journal of the Medical Library Association (JMLA), two from Health Information & Libraries Journal (HILJ) with the remaining one from the International Journal of Technology Assessment in Health Care (Int J Tech Ass in Hlth Care). Looking at the characteristics of all 87 cited studies we found that they mainly fell into four categories: Articles specifically from the health information literature (for example HILJ in both its manifestations yielded 20 different citations, narrowly pipped by BMLA/JMLA with 21 references and with Medical Reference Services Quarterly a long way back in third place (three citations). Articles from the general information literature (such as College and Research Libraries, Electronic Library, Program, Journal of Documentation). Articles from the general medical literature (BMJ yielded seven citations with isolated mentions for such journals as MJA and JAMA). Articles from the specialty health literature (e.g. Australian Journal of Midwifery, British Journal of Community Nursing, British Dental Journal, Contraception, Psychiatric Review). This illustrates the breadth of the health information evidence base, which is supplemented by a fifth category—unpublished studies including conference proceedings. Two particular studies had more than one citation (the DISCERN project on consumer health information—in both a health information and a general health journal—and a study at Glasgow Caledonian University—again in a health information journal, but this time also in a general information journal). The oldest citation dated from 1981, being the first recorded randomized controlled trial in health librarianship. One author was cited three times (three different articles), but as he was the facilitator of the FOLIO course this may have been in the mistaken belief that it would elicit more favourable marking (it didn't!). Interestingly, one of those three citations was to a previous article in this research column involving a secondary analysis of primary studies by other authors. This provides circumstantial evidence of a role for research synthesis in hastening application of research into practice While the above represents an opportunistic, non-systematic survey of librarians’ perceptions of the usefulness of research, it does support several complementary purposes. At a simple level, it indicates the interests of practising librarians and the sources that they use to populate their evidence base. It also attests to the pivotal role of HILJ and JMLA in communicating research findings, supported by a secondary level of general information journals and general medical and specialty journals as they increasingly reflect interest in information issues. Of concern, however, is whether the health information literature is currently best equipped to handle all four categories of research application as identified above. In particular, if much research has to be either locally validated or is used to inform the development of methodologies or instruments, is sufficient detail provided to sustain these purposes? Does the health information literature, which is primarily descriptive, also facilitate the level of critical analysis required either for local adaptation or to inform a broader understanding of issues as necessitated by ‘enlightenment’? Clearly, as a profession, we need to develop the supporting machinery of evidence-based practice such as explicit reporting of methods, validation of instruments and tools, and a more pervasive approach to critical appraisal and systematic review. After all we ‘just never know what will happen next!’