Comparing the effectiveness of a medication knowledge base product as designed with real-world hospital implementations using the Leapfrog Group’s Computerized Physician Order Entry (CPOE/EHR) Evaluation Tool

Objectives We sought to compare and analyze the performance of a medication knowledge base (MKB) vendor’s (Medi-Span) fully enabled, factory settings product to operational implementations in hospitals using the Leapfrog Group’s Computerized Physician Order Entry (CPOE)/electronic health record (EHR) Evaluation Tool. Materials and Methods We randomly selected a hospital that used the MKB vendor’s product to complete the Leapfrog CPOE/EHR Evaluation Tool in 2018 and used this version of the tool to assess the MKB vendor’s fully enabled product. Next, we retrospectively evaluated all hospitals, regardless of EHR vendor, that used the MKB vendor’s product in the tool from 2017 to 2019. We fit a multivariate linear regression model with hospitals’ overall scores as the dependent variable and EHR vendor as the independent variable. Results The fully enabled version of the MKB vendor achieved a 94% overall score in the tool. The mean overall score across 358 hospitals from 2017 to 2019 ranged from 61.1% to 68.8%. Finally, the regression model revealed EHR Vendor B hospitals scored 3.6 percentage points lower (P = .01) than EHR Vendor A hospitals, but the model only explained 3.7% of the variation in overall scores. Discussion The study revealed variations in the effectiveness of implementation and configuration of the MKB vendor’s product across all EHR vendors at the hospital level. Conclusion Our regression model found that EHR vendor had minimal effect on the overall tool scores. Therefore, EHR vendor is not nearly as important in the performance on the CPOE/EHR Evaluation Tool as the choices made in configuring the MKB vendor’s product within a specific EHR.