Towards a national perioperative clinical quality registry: The diagnostic accuracy of administrative data in identifying major postoperative complications

Accurately measuring the incidence of major postoperative complications is essential for funding and reimbursement of healthcare providers, for internal and external benchmarking of hospital performance and for valid and reliable public reporting of outcomes. Actual or surrogate outcomes data are typically obtained by one of three methods: clinical quality registries, clinical audit, or administrative data. In 2017 a perioperative registry was developed at the Alfred Hospital and mapped to administrative and clinical data. This study investigated the statistical agreement between administrative data (International Statistical Classification of Diseases and Related Health Problems (10th edition) Australian Modification codes) and clinical audit by anaesthetists in identifying major postoperative complications. The study population included 482 high-risk surgical patients referred to the Alfred Hospital anaesthesia postoperative service over two years. Clinical audit was conducted to determine the presence of major complications and these data were compared to administrative data. The main outcome was statistical agreement between the two methods, as defined by Cohen's kappa statistic. Substantial agreement was observed for five major complications, moderate agreement for three, fair agreement for six and poor agreement for two. Sensitivity and positive predictive value ranged from 0 to 100%. Specificity was above 90% for all complications. There was important variation in inter-rater agreement. For four of the five complications with substantial agreement between administrative data and clinical audit, sensitivity was only moderate (61.5%-75%). Using International Statistical Classification of Diseases and Related Health Problems (10th edition) Australian Modification codes to identify postoperative complications at our hospital has high specificity but is likely to underestimate the incidence compared to clinical audit. Further, retrospective clinical audit itself is not a highly reliable method of identifying complications. We believe a perioperative clinical quality registry is necessary to validly and reliably measure major postoperative complications in Australia for benchmarking of hospital performance and before public reporting of outcomes should be considered.