Data provenance and integrity of health-care systems data for clinical trials

The need to run clinical trials quickly and efficiently is well recognised, none more so than during the push to find life-saving treatments and preventative measures for COVID-19. Late phase clinical trials can take many years and are expensive, due to the immense efforts necessary to deliver them. There are various ways in which the conduct of clinical trials could be improved, one is through judicious use of data already collected in health-care interactions. These data might be known as health-care systems data, routinely collected health-care data (RCHD), or real-world data. We describe here how one key roadblock to the use of these data can be removed. In the UK, 50% of the National Institute for Health and Care Research-funded trials (to 2019) were planning to access and use RCHD. 1McKay AJ Jones AP Gamble CL Farmer AJ Williamson PR Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials. F1000 Res. 2020; 9: 323Crossref Scopus (4) Google Scholar However, looking at the data given to trials from registries between 2013 and 2018, fewer than 5% of UK-based randomised clinical trials obtained RCHD. 2Lensen S Macnair A Love SB et al. Access to routinely collected health data for clinical trials – review of successful data requests to UK registries. Trials. 2020; 21: 398Crossref Scopus (9) Google Scholar As notable examples, the RECOVERY3Horby P Lim WS Emberson JR et al. Dexamethasone in hospitalized patients with Covid-19. N Engl J Med. 2021; 384: 693-704Crossref PubMed Scopus (4803) Google Scholar and PRINCIPLE4Yu LM Bafadhel M Dorward J et al. Inhaled budesonide for COVID-19 in people at high risk of complications in the community in the UK (PRINCIPLE): a randomised, controlled, open-label, adaptive platform trial. Lancet. 2021; 398: 843-855Summary Full Text Full Text PDF PubMed Scopus (91) Google Scholar trials of potential treatments for COVID-19 have each successfully harnessed benefits of RCHD, which has simplified multi-site data collection through centralised collation and aided identification of potential trial participants. 5Pinches H NHS DigiTrials: how the search for COVID-19 treatments is revolutionising clinical trials. The Pharmaceutical Journal. 2021; 306DOI: 10. 1211/PJ. 2021. 1. 75436Google Scholar There is an intention by trialists more widely to make use of RCHD for study design and recruitment through to outcome ascertainment and post-trial follow-up;1McKay AJ Jones AP Gamble CL Farmer AJ Williamson PR Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials. F1000 Res. 2020; 9: 323Crossref Scopus (4) Google Scholar, 2Lensen S Macnair A Love SB et al. Access to routinely collected health data for clinical trials – review of successful data requests to UK registries. Trials. 2020; 21: 398Crossref Scopus (9) Google Scholar, 5Pinches H NHS DigiTrials: how the search for COVID-19 treatments is revolutionising clinical trials. The Pharmaceutical Journal. 2021; 306DOI: 10. 1211/PJ. 2021. 1. 75436Google Scholar, 6Sydes MR Barbachano Y Bowman L et al. Realising the full potential of data-enabled trials in the UK: a call for action. BMJ Open. 2021; 11e043906Crossref Scopus (3) Google Scholar successful high-profile example trials will encourage further uptake. Regulatory issues are a major challenge to wider use of RCHD in trials. The Medicines and Healthcare products Regulatory Agency in the UK and the US Food and Drug Administration recognise the potential value of RCHD for clinical trials that support regulatory decisions and each published draft guidance. Trial sponsors nevertheless need to demonstrate that all the data used in the trial, including the RCHD, are reliable, complete, and relevant. This involves assessing data provenance and integrity, and the validity (diagnostic value) and suitability of routine datasets for trial measures (eg, outcomes, exposures, and covariates). 7Medicines and Healthcare products Regulatory Agency'GXP' Data Integrity Guidance and Definitions. https: //assets. publishing. service. gov. uk/government/uploads/system/uploads/attachmentdata/file/687246/MHRAGxPdataᵢntegrityguideMarchₑditedFinal. pdfDate: 2018Date accessed: February 10, 2021Google Scholar Data provenance is the detailed record of the origins of the data, the processes, and the methods by which it is produced. Data integrity is defined as the extent to which all data are complete, consistent, accurate, and reliable throughout the data lifecycle. 8Clinical Data Interchange Standards Consortium Electronic Source Data Interchange GroupLeveraging the CDISC standards to facilitate the use of electronic source data within clinical trials. https: //www. cdisc. org/system/files/all/referenceₘaterialcategory/application/pdf/esdi. pdfDate: 2006Date accessed: September 29, 2020Google Scholar Although there is a standard endorsed by regulators for assessing systems that create or capture electronic clinical data as source (that is, original records), 9Murray ML Pinches H Mafham M et al. Use of NHS Digital datasets as trial data in the UK: a position paper. Zenodo. 2022; (published online Feb 11. ) https: //www. doi. org/10. 5281/zenodo. 6047155Google Scholar there has been insufficient guidance on how to assess centrally curated RCHD. Therefore, we developed a process to ascertain and document the provenance and integrity of RCHD. Our in-depth report has recently been made publicly available. 9Murray ML Pinches H Mafham M et al. Use of NHS Digital datasets as trial data in the UK: a position paper. Zenodo. 2022; (published online Feb 11. ) https: //www. doi. org/10. 5281/zenodo. 6047155Google Scholar This report sets out the methods which we applied to the two NHS Digital data assets most requested by trialists: the Admitted Patient Care dataset of Hospital Episode Statistics (HES APC) and the Civil Registration of Deaths (CRD). 9Murray ML Pinches H Mafham M et al. Use of NHS Digital datasets as trial data in the UK: a position paper. Zenodo. 2022; (published online Feb 11. ) https: //www. doi. org/10. 5281/zenodo. 6047155Google ScholarThe provenance and integrity of these two datasets were evaluated in three key stages: first, collection and transfer of data from health-care systems to NHS Digital's systems; second, centralised processing and curation to form the validated dataset; and, finally, linkage and extraction for trialists and the sponsor. At each stage, we reviewed the tools and systems used, and the controlled processes for managing data, data lineage, and access arrangements. Advice about the level of detail required for documentation was sought throughout from the Medicines and Healthcare products Regulatory Agency, who provided helpful feedback through the development process. By investigating the data lifecycles of HES APC and CRD, we have demonstrated that their curation is robust, and handled with appropriate controls and automation. We are confident that the data can be considered as equivalent to high-quality transcribed versions of the original source data, and so are sufficiently reliable for use in clinical trials. Our detailed approach has clear implications for the design, conduct, and analysis of clinical trials. We have demonstrated that these two key health-care systems datasets have the provenance and data integrity for use in clinical trials that would be suitable in regulatory submissions. This approach is relevant to industry as well as academia. Greater use of RCHD should change many aspects of trial conduct, including the way trials are monitored and in particular, probably decreasing their carbon footprint. Our work on two health-care systems datasets is only the initial step. The integrity and provenance of each routinely collected dataset that might be used in clinical trials should now be systematically assessed and clearly documented using the same approach. We call upon, and strongly encourage, all data collators to share and maintain the necessary documentation in a similar manner that we have started for HES APC and CRD. Trialists must also record the relevance of RCHD (validity and suitability) in their trial protocol and Trial Master File, and we suggest a process of curation and documentation of these choices. 10The Healthcare Systems Data for Clinical Trials Collaborative GroupRoutine dataset justification template. Zenodo. 2022; (published online Feb 15. ) https: //doi. org/10. 5281/zenodo. 6047938Google Scholar Further work is important to assess the use of RCHD against traditional trial-specific data collection methods so trialists can choose which approach to use for data items, accounting for availability, completeness, timeliness, latency, and cost. Such assessments can be achieved through studies-within-a-trial in existing trials. In conclusion, RCHD has the potential to transform the conduct of clinical trials, but their sponsors need confirmation of their integrity and provenance to satisfy regulatory-grade standards. We have demonstrated the process for two important datasets and now urge data providers to take the necessary steps to facilitate this for all relevant datasets. These steps will make trials more efficient and consequently lead to faster improvements in health care for all. The need to run clinical trials quickly and efficiently is well recognised, none more so than during the push to find life-saving treatments and preventative measures for COVID-19. Late phase clinical trials can take many years and are expensive, due to the immense efforts necessary to deliver them. There are various ways in which the conduct of clinical trials could be improved, one is through judicious use of data already collected in health-care interactions. These data might be known as health-care systems data, routinely collected health-care data (RCHD), or real-world data. We describe here how one key roadblock to the use of these data can be removed. In the UK, 50% of the National Institute for Health and Care Research-funded trials (to 2019) were planning to access and use RCHD. 1McKay AJ Jones AP Gamble CL Farmer AJ Williamson PR Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials. F1000 Res. 2020; 9: 323Crossref Scopus (4) Google Scholar However, looking at the data given to trials from registries between 2013 and 2018, fewer than 5% of UK-based randomised clinical trials obtained RCHD. 2Lensen S Macnair A Love SB et al. Access to routinely collected health data for clinical trials – review of successful data requests to UK registries. Trials. 2020; 21: 398Crossref Scopus (9) Google Scholar As notable examples, the RECOVERY3Horby P Lim WS Emberson JR et al. Dexamethasone in hospitalized patients with Covid-19. N Engl J Med. 2021; 384: 693-704Crossref PubMed Scopus (4803) Google Scholar and PRINCIPLE4Yu LM Bafadhel M Dorward J et al. Inhaled budesonide for COVID-19 in people at high risk of complications in the community in the UK (PRINCIPLE): a randomised, controlled, open-label, adaptive platform trial. Lancet. 2021; 398: 843-855Summary Full Text Full Text PDF PubMed Scopus (91) Google Scholar trials of potential treatments for COVID-19 have each successfully harnessed benefits of RCHD, which has simplified multi-site data collection through centralised collation and aided identification of potential trial participants. 5Pinches H NHS DigiTrials: how the search for COVID-19 treatments is revolutionising clinical trials. The Pharmaceutical Journal. 2021; 306DOI: 10. 1211/PJ. 2021. 1. 75436Google Scholar There is an intention by trialists more widely to make use of RCHD for study design and recruitment through to outcome ascertainment and post-trial follow-up;1McKay AJ Jones AP Gamble CL Farmer AJ Williamson PR Use of routinely collected data in a UK cohort of publicly funded randomised clinical trials. F1000 Res. 2020; 9: 323Crossref Scopus (4) Google Scholar, 2Lensen S Macnair A Love SB et al. Access to routinely collected health data for clinical trials – review of successful data requests to UK registries. Trials. 2020; 21: 398Crossref Scopus (9) Google Scholar, 5Pinches H NHS DigiTrials: how the search for COVID-19 treatments is revolutionising clinical trials. The Pharmaceutical Journal. 2021; 306DOI: 10. 1211/PJ. 2021. 1. 75436Google Scholar, 6Sydes MR Barbachano Y Bowman L et al. Realising the full potential of data-enabled trials in the UK: a call for action. BMJ Open. 2021; 11e043906Crossref Scopus (3) Google Scholar successful high-profile example trials will encourage further uptake. Regulatory issues are a major challenge to wider use of RCHD in trials. The Medicines and Healthcare products Regulatory Agency in the UK and the US Food and Drug Administration recognise the potential value of RCHD for clinical trials that support regulatory decisions and each published draft guidance. Trial sponsors nevertheless need to demonstrate that all the data used in the trial, including the RCHD, are reliable, complete, and relevant. This involves assessing data provenance and integrity, and the validity (diagnostic value) and suitability of routine datasets for trial measures (eg, outcomes, exposures, and covariates). 7Medicines and Healthcare products Regulatory Agency'GXP' Data Integrity Guidance and Definitions. https: //assets. publishing. service. gov. uk/government/uploads/system/uploads/attachmentdata/file/687246/MHRAGxPdataᵢntegrityguideMarchₑditedFinal. pdfDate: 2018Date accessed: February 10, 2021Google Scholar Data provenance is the detailed record of the origins of the data, the processes, and the methods by which it is produced. Data integrity is defined as the extent to which all data are complete, consistent, accurate, and reliable throughout the data lifecycle. 8Clinical Data Interchange Standards Consortium Electronic Source Data Interchange GroupLeveraging the CDISC standards to facilitate the use of electronic source data within clinical trials. https: //www. cdisc. org/system/files/all/referenceₘaterialcategory/application/pdf/esdi. pdfDate: 2006Date accessed: September 29, 2020Google Scholar Although there is a standard endorsed by regulators for assessing systems that create or capture electronic clinical data as source (that is, original records), 9Murray ML Pinches H Mafham M et al. Use of NHS Digital datasets as trial data in the UK: a position paper. Zenodo. 2022; (published online Feb 11. ) https: //www. doi. org/10. 5281/zenodo. 6047155Google Scholar there has been insufficient guidance on how to assess centrally curated RCHD. Therefore, we developed a process to ascertain and document the provenance and integrity of RCHD. Our in-depth report has recently been made publicly available. 9Murray ML Pinches H Mafham M et al. Use of NHS Digital datasets as trial data in the UK: a position paper. Zenodo. 2022; (published online Feb 11. ) https: //www. doi. org/10. 5281/zenodo. 6047155Google Scholar This report sets out the methods which we applied to the two NHS Digital data assets most requested by trialists: the Admitted Patient Care dataset of Hospital Episode Statistics (HES APC) and the Civil Registration of Deaths (CRD). 9Murray ML Pinches H Mafham M et al. Use of NHS Digital datasets as trial data in the UK: a position paper. Zenodo. 2022; (published online Feb 11. ) https: //www. doi. org/10. 5281/zenodo. 6047155Google Scholar The provenance and integrity of these two datasets were evaluated in three key stages: first, collection and transfer of data from health-care systems to NHS Digital's systems; second, centralised processing and curation to form the validated dataset; and, finally, linkage and extraction for trialists and the sponsor. At each stage, we reviewed the tools and systems used, and the controlled processes for managing data, data lineage, and access arrangements. Advice about the level of detail required for documentation was sought throughout from the Medicines and Healthcare products Regulatory Agency, who provided helpful feedback through the development process. By investigating the data lifecycles of HES APC and CRD, we have demonstrated that their curation is robust, and handled with appropriate controls and automation. We are confident that the data can be considered as equivalent to high-quality transcribed versions of the original source data, and so are sufficiently reliable for use in clinical trials. Our detailed approach has clear implications for the design, conduct, and analysis of clinical trials. We have demonstrated that these two key health-care systems datasets have the provenance and data integrity for use in clinical trials that would be suitable in regulatory submissions. This approach is relevant to industry as well as academia. Greater use of RCHD should change many aspects of trial conduct, including the way trials are monitored and in particular, probably decreasing their carbon footprint. Our work on two health-care systems datasets is only the initial step. The integrity and provenance of each routinely collected dataset that might be used in clinical trials should now be systematically assessed and clearly documented using the same approach. We call upon, and strongly encourage, all data collators to share and maintain the necessary documentation in a similar manner that we have started for HES APC and CRD. Trialists must also record the relevance of RCHD (validity and suitability) in their trial protocol and Trial Master File, and we suggest a process of curation and documentation of these choices. 10The Healthcare Systems Data for Clinical Trials Collaborative GroupRoutine dataset justification template. Zenodo. 2022; (published online Feb 15. ) https: //doi. org/10. 5281/zenodo. 6047938Google Scholar Further work is important to assess the use of RCHD against traditional trial-specific data collection methods so trialists can choose which approach to use for data items, accounting for availability, completeness, timeliness, latency, and cost. Such assessments can be achieved through studies-within-a-trial in existing trials. In conclusion, RCHD has the potential to transform the conduct of clinical trials, but their sponsors need confirmation of their integrity and provenance to satisfy regulatory-grade standards. We have demonstrated the process for two important datasets and now urge data providers to take the necessary steps to facilitate this for all relevant datasets. These steps will make trials more efficient and consequently lead to faster improvements in health care for all. MM declares research grants from Novartis and Novo Nordisk unrelated to this manuscript. MJL declares grants from the Industrial Strategy Challenge Fund, HDR UK, National Institute for Health and Care Research Oxford Biomedical Research Centre, and Medical Research Council (MRC) Population Health Research Unit unrelated to this manuscript. MRS declares research grants from Astellas, Clovis Oncology, Janssen, Pfizer, Novartis, and Sanofi-Aventis unrelated to this manuscript; and speaker fees from Lilly Oncology and Janssen unrelated to this manuscript. SBL, MLM, JRC, SH, MKBP, and HP declare no competing interests. MLM, SBL, JRC, MKBP, and MRS acknowledge funding from HDR UK (HDR-9005) and MRC (MCUU₀0004/07, MCUU₀0004/08, and MCUU₀0004/09).