Recycling old data – an ethical and pragmatic way to generate new evidence?

In this issue of Anaesthesia, Ho et al. report data suggesting that, for cardiac surgical patients, platelet dysfunction towards the end of cardiopulmonary bypass is associated with adverse postoperative outcomes 1. Following corrections for several confounders, they conclude a 1% increase in platelet dysfunction was associated with a 1% increase in the odds of a composite outcome of adverse events. Before we consider the clinical implications of this finding, there are two important aspects of their paper that warrant further discussion. These relate to the stepped-wedge cluster randomised controlled trial from which the authors retrieved data to test their hypothesis 2 and the recycling of old datasets to answer new research questions. There is already good evidence that patient blood management strategies reduce peri-operative allogeneic blood transfusions and are associated with better postoperative outcomes for patients undergoing all types of major surgery 3, 4. It also seems reasonable that access to intra-operative point-of-care tests of coagulation in combination with integrated transfusion algorithms may reduce the need for allogeneic blood transfusion. That said, before the study by Karkouti et al. 2, the evidence for such strategies was limited to observational, non-controlled studies. In order to build a case for the purchasing of equipment, hospitals, departments, policymakers and clinical managers may ask for randomised controlled trial evidence. There are nevertheless several logistical barriers to a conventional trial such as, but not limited to, the complexity of ‘rolling out’ the intervention at different time-points for different patients at different centres. When the intervention is a complex package of care, such as those commonly employed in large multi-centre quality improvement projects 5, 6, the stepped-wedge design allows for randomisation of clusters at different time-points (Fig. 1). In stepped-wedge cluster trials, observations are first made during a baseline period when all clusters are controls, and no participant receives the intervention. Clusters then cross over from the control arm to the intervention arm, but at different time-points. As the study progresses, increasing numbers of clusters cross over to the intervention arm so that by the end of the study, all participants in all clusters are receiving the intervention. As this occurs, data are continually collected post-intervention. The name ‘stepped wedge’ comes from the stepped wedge shape in Fig. 1. The design of such studies is complicated by many factors, including: the need to determine the number of clusters; the length of each step; the number of clusters randomised at each step and how best to power such studies 7. There are also uncertainties around how such trials should be reported and interpreted, although a Consolidated Standards of Reporting Trials (CONSORT) extension was recently published 8. It is perhaps surprising that Anaesthesia has published so few stepped-wedge trials, given such studies seem to be emerging as a useful way in which to quantify the benefit from interventions aimed at improving quality 7, 9. We expect to see many more peri-operative stepped-wedge trials in the future and encourage readers to review the methods used in the study by Karkouti et al. 2, which is an excellent example of a well-conducted and reported stepped-wedge trial. Another interesting aspect of the study by Ho et al. is the use of raw data from a previously published clinical trial. Some may ask whether or not this is ‘salami slicing’, and the stance Anaesthesia takes in this regard is clear 10. Salami slicing is the breaking up of data into multiple publications and it distorts the literature by giving the impression that slices are derived from different source data. The Committee on Publication Ethics (COPE) guidance is that salami slicing is likely when studies from the same group test the same hypothesis using the same population and methods (https://publicationethics.org/case/salami-publication). This new work by Ho et al. 1 is not salami sliced, as the questions asked of their data are entirely different from the original study 2. That said, there remain several issues relating to secondary analyses that have not, we believe, been sufficiently addressed by publishers, journals, editors and researchers. The International Committee of Medical Journal Editors (ICMJE) provides guidance on overlapping publications (http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/overlapping-publications.html). They state that secondary analyses should cite any primary publications and clearly state the source of the dataset. The overall theme is one of transparency – secondary analyses are acceptable if they are seen to be open and transparent, and the population is relevant to the hypothesis. Despite the requirement for fresh research ethical committee approvals, conclusions may still be influenced by practices such as p-hacking, data dredging and primary outcome modification. Secondary analyses arguably become unacceptable, however, when they lack transparency, are salami sliced, or report simple subanalyses of previously analysed data. There is currently no consensus on the degree to which the conclusions of secondary analyses can be translated into clinical practice. Some may argue the only meaningful conclusions from the primary dataset relate to the outcome for the which the trial was powered, which in this case was red blood cell transfusion occurring from the start of surgery to the end of postoperative day seven (or hospital discharge if earlier) 2. Perhaps it is easier to discount this view if the primary trial is sufficiently large, although how large it needs to be is unclear. Some may also argue such conclusions should be used only to generate, revise and inform hypotheses, and should not be used to demonstrate causality. In that sense, there are many parallels to be drawn with observational database studies 11. The primary study by Karkouti et al. waivered consent for included patients 2, and there are good reasons for this. If, however, participants were consented for inclusion in a study to answer question A, it may not always be ethical to undertake secondary analyses of the same data to answer question B. It could be argued new research ethical committee approvals are not always sufficient in this regard, and a new cohort of consented patients might be required to answer question B. That said, in the era of ‘big data’, there are several arguments against such a restrictive approach to secondary analyses of old datasets. Healthcare organisations generate vast quantities of data but much is unanalysed, underutilised and destined for eternal redundancy. We can nevertheless only make sense of clinical practice if we understand underlying trends including what works and what fails 12. There is an urgent need to better utilise routinely collected clinical data but we can only do so if we ask the right questions, have access to the relevant analytical expertise, and if we are permitted access to datasets. Analysing routinely collected clinical data from a multi-centre randomised controlled trial is arguably no different. If there are clinically-relevant conclusions that can be drawn, it is ethical to ensure we maximise the use of such data. This adds value for patients participating in studies as well as for funders, researchers and all other stakeholders. The financial costs of such an approach are negligible, evidence can be generated more rapidly and research waste is reduced 13. Although the use of data in this way is by no means perfect, and, for example, is influenced by confounding 14 and temporal biases 15, we argue the secondary analysis conducted by Ho et al. is warranted, clinically-relevant and ethically justified. Pandit and Merry have argued the case for archiving clinical trial data on prepublication servers, and this would certainly help promote the sharing of data 13. Recently, however, several major orthopaedic journals stated they would automatically reject manuscripts previously posted to preprint servers 16. Some may criticise the study by Ho et al. for: its secondary analysis of clinical trial data; the use of a composite outcome 17; the lack of long-term follow-up; and a focus on only one type of point-of-care test, when others were conducted and may similarly be used as surrogates. For example, it may also be the case that hypofibrinogenaemia, as detected by FIBTEM, is a surrogate for bypass-associated haemostatic activation. Furthermore, the usefulness of risk stratification at this stage of surgery, when certain risk factors such as operation type or circulatory arrest time cannot be modified, must be brought into question. One possibility is that such tests may be a way to account for the difference between pre- and postoperative risk, which Coulson et al. previously termed the ‘acute risk change’ 18. Although this seems to be associated with postoperative mortality following cardiac surgery, Carlisle challenged this by suggesting their finding was a mathematical consequence of rounding the pre-operative but not the postoperative risk 19. Similarly, if one estimates the distribution of percentage platelet dysfunction from the median reported by Ho et al., it may appear as though the distributions are similar for those who did and did not have an event. Regardless, we argue Ho et al. have generated an interesting hypothesis that requires further study. We must also think carefully about the use of point-of-care tests of coagulation for risk stratification and how they might be integrated into routine clinical practice. MC is Social Media Editor of Anaesthesia. No other conflict of interest to declare.