609 Background: Radical cystectomy is a potentially morbid procedure that presents a prime opportunity for quality improvement and patient safety initiatives. We hypothesized that a novel electronic medical record (EMR)-based automated algorithm for surgical outcomes and quality metrics following radical cystectomy would demonstrate >90% sensitivity and specificity with significant inter-rater reliability (IRR) with institutional National Surgical Quality Improvement Program (NSQIP) data abstraction. Methods: We developed a novel EMR-based algorithm that automatically abstracts surgical outcomes and quality metrics for cystectomy cases to facilitate quality improvement initiatives. Surgical outcomes were extracted using CPT and ICD-10 codes and EMR-based variables; pathology results were extracted through text extraction. The sensitivity and specificity of algorithm and NSQIP was calculated; IRR between the algorithm and NSQIP abstraction was assessed using Cohen’s kappa statistic. Results: 634 cases were mutually tracked. Sensitivity of the algorithm was ≥ 90% for all outcomes; specificity was ≥ 96% for all outcomes. IRR was highest for mortality, stroke, and nodal stage (k=1.00) and lowest for ureteral obstruction, anastomotic leak, and rectal injury (k=0.00). IRR was slight for renal insufficiency (k=0.13) and pneumonia (k=0.19); fair for unplanned intubations (k=0.24), prolonged NPO (k=0.26), sepsis (k=0.28), and urinary leak (k=0.28); moderate for UTI (k=0.46), dialysis (k=0.58), and return to OR (k=0.60); and substantial for cardiac complications (k=0.65), prolonged vent (k=0.76), C. difficile infection (k=0.85), readmission (k=0.89), and tumor stage (k=0.90). Conclusions: This EMR-based algorithm for radical cystectomy outcomes achieves at least 90% sensitivity and specificity for all variables, matching or exceeding NSQIP abstraction. This algorithm demonstrates how surgical outcomes and pathology results can be made immediately accessible and automated, with potential reductions in cost and resource utilization. National quality improvement programs such as NSQIP may benefit from EMR-based automated algorithms for quality metric abstraction.
Palencia et al. (Wed,) studied this question.