Abstract Introduction The clinical relevance of pharmacists’ interventions (PIs) is complex to determine. The CLEO tool is a multidimensional scoring system to assess the relevance of PIs across three dimensions: clinical, economic, and organisational impact. Aim This study aimed to nationwide validate the CLEO tool by clinical pharmacists in German hospitals using structured and representative clinical cases. Method The German CLEO version was adapted to the German hospital setting and supplemented with practical examples. Fifty up-to-date cases from the inpatient setting were developed in a multistage process following the Identification, Situation, Background, Assessment, Recommendation structure. In the first round, each rater was randomly assigned 20 from the pool of 50 clinical cases, ensuring that all cases were evaluated by multiple raters. After a 14-day washout period, the same 20 cases were reassessed by the same raters. In the second round, all raters from the first round were invited again, and a subset who volunteered assessed another 20 cases after intensified training. Inter- and intra-rater reliability were calculated using Krippendorff’s α and the intraclass correlation coefficient (ICC). User feedback was collected through a 16-item questionnaire. Results A total of 79 pharmacists from 56 hospitals participated in the first round; 27 completed the second round as well. Inter-rater reliability was poor across all three CLEO dimensions (Krippendorff’s α < 0.67), both overall and among experienced clinical pharmacists. Intra-rater reliability was good for all dimensions (ICC 0.63–0.74), highest for the clinical dimension (0.74). Most raters (77%) needed less than one minute per case. Overall, the CLEO tool was perceived by users as appropriate, precise, acceptable and feasible (mean score 5.36; 7-point Likert scale; 1 = strongly disagree, 7 = strongly agree). Conclusion Since clinical pharmacy is still a developing discipline in German hospitals, differences in clinical practice and professional expertise complicate the evaluation of PIs. While intra-rater reliability was good, the validation of the CLEO tool in Germany did not achieve satisfactory inter-rater reliability. The CLEO tool may be useful within institutions with shared standards, but broader application across diverse settings in Germany requires additional training, further research and standardisation of clinical pharmacy services.
Berger et al. (Tue,) studied this question.