On impacts and other factors

There have been many articles and editorials written about the impact factor over the last two or three years — both in occupational therapy and other health-related journals. It is interesting to note that all these articles have expressed concern about the flawed nature of the impact factor and about how it is used. The articles highlight the fact that many professions, including occupational therapy, are concerned about the way the impact factor is used; it is seen as a problem for medical and health research in general (Editorial Board, 2005; Lankhorst & Franchignoni, 2001; Monastersky, 2005a,b; Swedlove, 2006; Walker, 2005; Walter, Bloch, Hunt & Fisher, 2003; World Association of Medical Editors, 2004). In another arena, on the list serve for the World Association of Medical Editors (WAME), there has been a great deal of discussion about how the impact factor is used and how this use affects research and researchers. This is of interest to occupational therapists, many of whom feel threatened because there are few Occupational Therapy Journals with an impact factor. What is an impact factor? In the policy statement from WAME on its website it is suggested that ‘Journal editors have the responsibility to educate their readers, authors, administrators, and their scientific community in general about impact factor and its relevance’ (World Association of Medical Editors, 2004). It was developed as a grading system in the early 1960s to identify which journals mattered the most to scientists (Monastersky, 2005b), and is calculated by dividing the number of times articles from a journal are cited in journals on the Institute of Scientific Information (ISI) database, by the number of articles published in that journal over the same period of two years (http://www.isinet.com/). To become part of this database involves an extensive process of evaluation, with very few new applications being accepted (Lankhorst & Franchignoni, 2001). The Australian Occupational Therapy Journal was unsuccessful in its first application in 2003, the reasons given being that there were not enough research articles funded by grants, and that the topic area of publication was already well covered. This is a ‘vicious circle’— as the impact factor already plays a role in academic appointments, tenure decisions and awarding of grants. As Fern Swedlove states in her editorial in the Canadian Journal of Occupational Therapy, ‘It is likely that more occupational therapy articles will continue to be sent to other non-occupational therapy journals for publication … which carry more recognition due to their impact factor’ (2006, p. 4). She goes on to point out that although this is a problem because practising occupational therapists are less likely to read these journals, it does have the advantage of exposing a wider audience to occupational therapy research. In addition anecdotal evidence suggests that university administrators are encouraging staff to submit their research to journals that have an impact factor. As Walker (2005) points out in her British Journal of Occupational Therapy editorial, publishing in a high-impact journal is important for academic staff evaluation. It is suggested in many of the articles that there are limitations relating to the impact factor: ‘The assumption of a positive link between citations and quality is ill-founded, in that we cite articles for diverse reasons, including to refer to research judged as suspect or poor’ (Walter et al., 2003, p. 280). In addition self-citations are not accounted for and review articles are cited more often than original research. It suggests that the paper is valued most highly if it is published in a high-impact journal, rather than being valued for what the paper actually says. Another problem which has been highlighted is that the impact factor may be affecting what research is being undertaken (Monastersky, 2005b), with studies of smaller fields or from different regions in the world being rejected by some journals because they would not attract sufficient citations. How do all these issues affect occupational therapy? Several occupational therapy journals are in the process of planning their application to ISI (Editorial Board, 2005; Swedlove, 2005) while at the next World Federation of Occupational Therapists Congress in Sydney, a meeting of journal editors is planned to discuss the issues relating to ISI and the impact factor. However, because of the difficulties associated with the impact factor, there have been moves by other professions to develop different methods to highlight important papers, for example the Faculty of 1000, which was launched in 2001 (Current Sciences Group, 2001). This uses a group of scientists to evaluate the most important biology papers they read each month from a series of journals (Monastersky, 2005a). Universities such as McMaster University in Canada and the University of Southern California are also developing new methods for ranking research papers (Monastersky, 2005a). Another process used at McMaster University is that each discipline identifies their top journals for tenure and promotion purposes, using a combination of impact factors and the profession's knowledge about the quality of the journal (M. Law, personal communication, 2005). It should also be understood that a poorly cited article may be of interest to many practising clinicians and therefore be more important than an article of interest to a small number of researchers who will cite it frequently in their publications. A journal must look after its readers, in our case, mostly occupational therapists; as stated in a recent editorial in the British Journal of Occupational Therapy, ‘We must agree the ultimate aim for the future is to disseminate high quality evidence for our practice by publishing in our own professional journal’ (Walker, 2005, p. 55). Despite the problems and flaws of the impact factor it is still used in a critical way for grants, academic promotions and tenure; therefore it is our duty as an editorial board to pursue the ISI application process once more.