Provider inaccuracy in assessing adherence and outcomes with newly initiated antiretroviral therapy

We studied the ability of providers to predict and estimate patient adherence to newly initiated highly active antiretroviral therapy (HAART). Nineteen providers referring 40 patients into an adherence study were surveyed. Widespread inaccuracy was found in providers’ adherence predictions and estimates. Therefore, HAART should not be withheld solely on provider predictions of adherence. Providers should not rely on their own assessments when attempting to determine if patients are adhering to therapy. Highly active antiretroviral therapy (HAART) dramatically improves HIV-related outcomes [1]. However, suboptimal adherence is associated with decreased HIV suppression [2,3] and probably leads to resistance [4,5]. Guidelines emphasize that ‘the strength of a recommendation for initiating therapy depends on … willingness to adhere’ [1]. Accordingly, the notion of withholding HAART in some patients has been debated [6]. Moreover, despite a lack of evidence of utility [7] providers use demographics and adherence assessments in deciding whether to prescribe HAART [8,9] There are thus important ramifications if a physician's assessment of adherence is inaccurate, because errors in predicting good adherence may promote resistance [5], and errors in predicting poor adherence may delay treatment unnecessarily. We designed this study to determine the accuracy of providers in both the predictions of their patient's future adherence and estimates of past adherence. We enrolled a prospective cohort at the University of Pennsylvania between February 1998 and November 1999 to assess the relationship between adherence and outcome in protease inhibitor naive patients with viral loads greater than 10 000 copies/ml newly started on nelfinavir, either 750 mg three times a day or 1250 mg twice a day, in standard combinations [1]. The study and findings have previously been described [10]. Nelfinavir adherence was tracked using microelectronic monitors (MEMS; APREX, Menlo Park, CA, USA). Using a structured telephone assessement, the principal investigator (R.G.) asked providers within 7 days of enrollment to predict whether their patients would become undetectable after 4 months and to predict the percentage adherence over 4 months. After 4 months, he asked providers to estimate the percentage adherence during the study. Adherence data were not available during the study. Clinically obtained data were available per usual care. Spearman's correlation coefficients and 95% confidence intervals (CI) were calculated for the association between the measured percentage adherence and both the providers’ predictions and estimates of adherence. Stratified analyses by nelfinavir dosing frequency (twice versus three times a day) were performed. The kappa statistic [11] and 95% CI were calculated for the agreement between providers’ predictions of viral load becoming undetectable and whether patients achieved it. The proportion of provider responses that were ‘off target’ by more than 10% was calculated on the basis of data suggesting better virological outcomes with 10% increments in adherence [3]. We used STATA 6.0 for all analyses (STATA Corporation, College Station, TX, USA). The study was approved by the Institutional Review Boards of the University of Pennsylvania and the Philadelphia Veterans’ Administration Medical Center. All patients gave written informed consent and providers gave verbal consent. Between February 1998 and July 1999, 59 patients were enrolled. Seventeen (29%) dropped out, and provider data collection was missed for two (3%), yielding a cohort of 40 patients. The characteristics have been described elsewhere [10]. The viral load became undetectable for 25 patients (63%). Nineteen providers were included, with a median age of 38 years (range 31–60). Ten were women (53%); 15 were physicians (79%; three were fellows), and four were nurse-practitioners (21%). All providers were white. The median number of patients referred per provider was two (range 1–7). Six patients were excluded from the analysis of provider estimates because the questionnaire was not administered or providers were unavailable within 7 days of study closeout. There was no correlation between predicted and actual adherence (Table 1, row 1). No substantive differences were found by dose frequency. Providers erred in their prediction both by overestimating and underestimating, although underestimating was somewhat more prevalent: 22 out of 40 predictions (55%) were underestimates, 15 out of 40 (38%) were overestimates, and three (8%) were exact. Overall, 26 out of 40 predictions (65%) missed by more than 10%. Physicians’ accuracy of predicting undetectable viral loads could not be said to differ from chance agreement (Table 1, row 2). This result was unchanged, with dropouts included as if they were detectable (data not shown).Table 1: Accuracy of provider assessments and correlation with measured outcome [Caption for P11881] In contrast, there was a positive correlation between providers’ estimates at the end of the study and actual adherence (Table 1, row 3). Accordingly, providers were somewhat more accurate in their post-hoc estimation of adherence. Of the estimates, 15 out of 34 (44%) were overestimates, 17 out of 34 (50%) were underestimates, and two out of 34 (6%) were exact. Overall, 16 out of 34 (47%) of the estimates missed by more than 10%. These data demonstrate that providers are inaccurate both in predicting and estimating adherence and predicting outcomes after newly starting HAART, although predictions were worse than post-hoc estimates. Although our conclusion is similar, our study differs from that of Paterson et al. [3], who evaluated subjects in various stages of treatment (i.e. already on or newly starting). Therefore, they combined prediction and estimation in their evaluation of providers. In contrast, we concluded that providers are both poor predictors of adherence and poor estimators of adherence because we analysed these two activities separately. Given the strong relationship between adherence and outcome [2,3,10] these results have important implications. Given providers’ inaccuracies about adherence and the evidence that groups with special challenges can have treatment success [2], HAART should not be withheld solely on a provider's assessment of adherence potential. Furthermore, as providers are inaccurate in screening for poor adherence, further studies of both the risk factors for poor adherence and interventions are needed to identify higher-risk patients and provide assistance. This study has several limitations. The results may not be generalizable to all providers and regimens. Also, it was performed before the magnitude of adherence needed to become undetectable was characterized [2,3,10]. It would be interesting to determine if providers are now better estimators of adherence given these publications. However, the conclusions regarding providers’ predictions of adherence are unlikely to change. In conclusion, providers must not be overly swayed by their assessments of patients’ adherence in deciding whether to offer HAART. Instead, we must continue research into clinically usable predictive instruments (e.g. questionnaires) to use in place of providers’ judgement. Finally, methods of measuring adherence including self-reports and microelectronic monitors should be considered in clinical practice to replace providers’ clinical judgement [2,3,10]. Robert Grossa,b,c Warren B. Bilkerb,c Harvey M. Friedmana James C. Coyned Brian L. Stromb,c Acknowledgements The authors wish to thank the providers and their patients for participating in this study. They would also like to acknowledge the support of the PENN Infectious Diseases Clinical Trials Unit Nurses who were crucial to the parent study and to the PENN Center for AIDS Research.