The quality of systematic reviews

This is the authors' abstract. We don't add key points for this paper.

# Review is biased {#article-title-2} EDITOR—In discussing meta-analyses in the treatment of asthma, Jadad et al state that most reviews published in peer reviewed journals or funded by industry have serious methodological flaws.1 This summary is misleading and could have been put more succinctly as, “most reviews published in peer reviewed journals have serious methodological flaws,” since the industry reviews in their paper were a (similar) subset of the published papers. Jadad et al are, however, right in drawing attention to the inherent bias in their Cochrane approved quality index. Cochrane reviews are excellent in many respects but grossly deficient in at least one—namely, the reliance on software, RevMan, that is incapable of satisfying an essential and elementary requirement placed by drug regulators on sponsors, that “the particular model chosen should reflect the state of medical knowledge about the variables to be analysed as well as the statistical design of the trial.”2 RevMan cannot deal appropriately with covariates nor with multicentre, cluster randomised, minimised, or crossover trials. It is thus a suitable tool for single centre, randomised, parallel group trials in which no covariates are measured: a type of trial that is rather rare in my experience. On the other hand, the pharmaceutical industry has long employed doctors and statisticians capable of dealing with the complications of real clinical trials. For a good illustration, see the paper by Richardson and Bablok,3 which Jadad et al did not include. The biased and one sided review by Jadad et al cannot be taken as showing the superiority of Cochrane reviews to pharmaceutical industry reviews. 1. 1.↵1. Jadad AR, 2. Moher M, 3. Browman GP, 4. Booker L, 5. Sigouin C, 6. Fuentes M, 7. et al .Systematic reviews and meta-analyses on treatment of asthma: critical evaluation.BMJ2000;320: 537–540. (26 February.) [OpenUrl][1][Abstract/FREE Full Text][2] 2. 2.↵1. International Conference on Harmonisation Statistical .Principles for clinical trials.Statist Med1999;18:1905–1942. [OpenUrl][3] 3. 3.↵1. Holgate ST <!-- --> 1. Richardson W, 2. Bablok B .Clinical experience with formoterol in adults. In: Holgate ST ed.Formoterol: fast and long-lasting bronchodilation.London: Royal Society of Medicine Services,1992:23–37. # High quality reporting of both randomised trials and systematic reviews should be priority {#article-title-6} EDITOR—Senn1 [previous letter] is critical of the conclusion of Jadad et al in their critical evaluation that “most reviews published in peer reviewed journals or funded by industry … [1]: {openurl}?query=rft.jtitle%253DBMJ%26rft.stitle%253DBMJ%26rft.issn%253D0007-1447%26rft.aulast%253DJadad%26rft.auinit1%253DA.%2BR%26rft.volume%253D320%26rft.issue%253D7234%26rft.spage%253D537%26rft.epage%253D540%26rft.atitle%253DSystematic%2Breviews%2Band%2Bmeta-analyses%2Bon%2Btreatment%2Bof%2Basthma%253A%2Bcritical%2Bevaluation%26rft_id%253Dinfo%253Adoi%252F10.1136%252Fbmj.320.7234.537%26rft_id%253Dinfo%253Apmid%252F10688558%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [2]: /lookup/ijlink?linkType=ABST&journalCode=bmj&resid=320/7234/537&atom=%2Fbmj%2F321%2F7256%2F297.atom [3]: {openurl}?query=rft.jtitle%253DStatist%2BMed%26rft.volume%253D18%26rft.spage%253D1905%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx