Commentary: Strengthening the reporting of observational epidemiology the STROBE statement

We welcome Ebrahim and Clarke's comments1 on the STROBE Statement and are grateful for the opportunity to clarify some of the issues they raise in their editorial. What is STROBE all about? The STROBE Statement is a checklist of items that should be addressed in articles reporting cohort studies, case–control studies or cross-sectional studies, to STrengthen the Reporting of OBservational studies in Epidemiology. A short paper that presents the checklist and explains how it was developed will be published in several journals2 in October 2007, and will be freely available on the websites of these journals (see www.strobe-statement.org for links to the paper). The intention is to provide guidance on how to report observational research well: the recommendations are not prescriptions for designing or conducting studies—these decisions must be made by investigators who know the subject matter. Also, while clarity of reporting is a prerequisite to evaluation, the checklist should not be seen as an instrument to evaluate the quality of observational research. Good reporting does not necessarily mean good research. The importance of good reporting is that others—readers, fellow scientists, reviewers and editors—can form an informed opinion on whether the research was appropriate and what aspects might need more scrutiny. Yes, Ebrahim and Clarke are right: some of the recommendations included in STROBE are fairly basic. In this context, it is important to define the audience for whom STROBE is intended: the recommendations are predominantly aimed at those who use epidemiologic study designs without being expert epidemiologists. We think that they may well outnumber experienced and well-trained research epidemiologists. For example, studies indexed with the Medical Subject Heading ‘cohort studies’ in Medline are mainly published in clinical specialist journals, and originate from clinical departments (Table 1). Fundamental deficiencies in reporting have been identified for such journals. For example, a review of survival analyses published in cancer journals found that almost half of articles did not give any summary of length of follow-up.3 But the problem is not restricted to clinical specialist journals: a survey of recent practice in the reporting of epidemiological research4,5 included all major epidemiological and general medical journals and found, for example, that few investigators explained their choice of confounding variables. Asking investigators to ‘Make clear which confounders were adjusted for and why they were included’ might thus not only be pertinent to the example of folic acid and the risk of stroke mentioned by Ebrahim and Clarke, but an important issue in many other reports of epidemiological studies. So, although this and other STROBE recommendations ‘might be found in an epidemiology text targeted at Masters students in the first term of their first year’1 they are sorely needed to improve the reporting of epidemiological research. Journals that published the 100 most recent articles indexed in Medline as ‘cohort studies’, August 2007 The figures in brackets indicate the number of publications. Journals that published the 100 most recent articles indexed in Medline as ‘cohort studies’, August 2007 The figures in brackets indicate the number of publications. In addition to the short paper mentioned earlier, a detailed companion paper, the STROBE Explanation and Elaboration article, justifies the inclusion of the different checklist items and gives methodological background and published examples of what we consider transparent reporting. This explanatory paper,6 which will be published (also with immediate open access) in Epidemiology, and electronically in PLoS Medicine and Annals of Internal Medicine, is an integral part of STROBE. Some of the examples we used in the explanatory paper came from studies of lower methodological quality, whose results were never replicated—yet some aspects of the study were clearly reported. Again, good reporting does not necessarily mean good research. We strongly recommend using the STROBE checklist in conjunction with the explanatory article.6 Indeed, this article addresses many of the points raised by Ebrahim and Clarke.1 For example, they fear that investigators seeking guidance might be confused when asked to ‘Explain the scientific background and rationale for the investigation being reported’, because in some studies the original rationale for the study might have been very different from the purpose of the analysis the investigators aim to publish today. We explicitly address this situation in the explanatory paper, and advise authors to briefly restate the original aims of the study: this might help readers understand the context of the research and possible limitations in the data. We stress that the secondary use of existing data is a creative part of observational research and does not necessarily make results less credible or less important. For example, the Physicians’ Health Study, a randomized controlled trial of aspirin and carotene, was later used to confirm that a point mutation in the factor V gene was associated with an increased risk of venous thrombosis, but not myocardial infarction or stroke.7 STROBE asks authors to ‘Give a cautious overall interpretation of results, considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence’, in line with Richard Doll's important statement (cited by Ebrahim and Clarke1) on the need to confirm unexpected results with potential implications for public health in further studies. The need for replication, which is an important point in science in general,8 is well taken, but has little to do with good reporting of an individual study: it is not the responsibility of the scientists who report that study. Nevertheless, in the explanatory paper,6 we discuss the scope of observational studies, from reporting a first hint of a potential cause of a disease, to verifying the magnitude of previously reported associations and stress that further studies to confirm or refute initial observations are often needed.9 STROBE tries to accommodate these diverse uses of observational research—from discovery to refutation or confirmation. As the great mathematician, physicist and philosopher Jules Henri Poincaré (1854–1912) said: ‘Science is built up of facts, as a house is built of bricks; but an accumulation of facts is no more a science than a heap of bricks is a house.’10 Does this mean authors should be asked to ‘conduct a systematic review of other similar studies’?1 As a previous editorial in the International Journal of Epidemiology argued,11 systematic reviews should be seen as original research and be published as such, rather than be reported in a paragraph of a discussion section. Interestingly, The Lancet recently updated their policy, asking authors of randomized trials to illustrate the relation between existing and new evidence by referring to a systematic review and meta-analysis.12 We believe that in many situations this requirement is also appropriate for reports of observational research. But note that both The Lancet and the CONSORT recommendations for the reporting of randomized trials (Consolidated Standards of Reporting Trials)13 stop short of asking authors to do a systematic review and meta-analysis. Contrary to Ebrahim and Clarke's assertion, we do not think that observational studies are ‘largely incapable of making definitive conclusions on the basis of robust findings’1: there are situations where observational research is as valid, more appropriate and more informative than randomized trials.14–17 Observational research is important and often hugely successful—much of health care and public health depends on it, from genetics to infectious diseases, from environmental exposures to the prognostic stratification of patients. However, like all research, in all branches of science, results need informed, critical discussion and such discussion is only possible if authors report transparently what was done and why it was done. We share Ebrahim and Clarke's optimism that STROBE will make an important contribution to improving the quality of reporting of observational research. Finally, we stress that STROBE and other recommendations on the reporting of research should be seen as evolving documents that require continual assessment, refinement, and, if necessary, change.18 We will revise the checklist in the future, taking into account criticism,1 new evidence, and experience from its use. We invite readers to submit their comments via the STROBE website (www.strobe-statement.org). The authors are members of the STROBE group.