Predicting the long‐term outcomes of biologics in patients with psoriasis using machine learning

Dear Editor, Despite the vast amount of data on the efficacy of biologics in psoriasis treatment, therapeutic decision making is still based on a trial‐and‐error approach. In a real‐world setting over 50% of patients needing dose adjustment during therapy and 20–50% of patients experiencing a relapse of the disease requiring a switch to another medication.1,2,3,4 We believe that any model that accurately predicts the long‐term efficacy of biologics would need to accommodate complex patterns of interactions between numerous variables. Machine learning techniques are often able to detect such complex patterns and are being increasingly used with success in predicting future trajectories of patients’ health in diverse areas such as cancer,5 diabetic complications6 and cardiovascular mortality.7 We examined here whether machine learning could aid in predicting long‐term responses to biologics in psoriasis through analysis of data of 681 patients with psoriasis from the Danish registry cohort, DERMBIO. We considered six different modelling techniques, which span a range of different prediction approaches: (i) generalized linear model (GLM), (ii) support vector machine, (iii) decision tree, (iv) random forest, (v) gradient‐boosted trees and (vi) deep learning. We applied the different machine learning algorithms on the same dataset to evaluate the performance of different approaches and to select the learner that produced the model that performed the best in terms of accuracy and runtime in a multinomial classification problem. Patient variables analysed included general demographics (age, sex, height, weight, number of comorbidities), disease specifics [age at diagnosis, presence of psoriatic arthritis, previous or concurrent methotrexate, previous biologic use, and baseline Dermatology Life Quality Index (DLQI) and Psoriasis Area and Severity Index (PASI)] and therapy specifics (name of biologic initiated, treatment series in same patient). Therapy outcome measures included length of treatment series and causes of drug discontinuation at the last observation, categorized as lack of drug efficacy, adverse event(s) or other (patient's decision, lost to follow‐up, not otherwise specified). Drug discontinuation was a surrogate measure of treatment failure. Based on a training dataset of 681 patients with psoriasis and using fivefold‐balanced cross‐validation,8 each of the six algorithms predicted drug discontinuation with high accuracy, ranging from 0·73 ± 0·06 to 0·82 ± 0·03. We were thus able to predict the treatment outcomes with < 18% classification error, simply utilizing basic health information routinely available to clinicians. As GLM outperformed other learners in terms of accuracy, we also plotted the sensitivity and specificity achieved by this algorithm as a receiver operating characteristic curve and calculated its area under the curve to determine the performance of GLM in multiclass classification. Figure 1 shows that GLM performs better in classifying the reason for discontinuation due to ‘any reason’, ‘lack of efficacy’, ‘adverse event’ and ‘other reasons’. In Figure 1(a) a curve closer to the top left corner and in Figure 1(b) a high value of area under the curve indicate better classification performance. In addition to this (classification) GLM model for predicting the overall drug discontinuation, we also produced another (regression) GLM model to calculate the expected length of treatment. This too was accurate, with a mean absolute error of 4·5 months. Multiple‐class receiver operating characteristic and area under the curve (AUC) analysis for the generalized linear model. (a) The curves show the performance of the model for predicting drug discontinuation due to any reason, adverse event, lack of efficacy, or other reasons (patient decision, loss to follow‐up, not specified). (b) AUC values for different causes of drug discontinuation. Input optimization (or prescriptive analytics) in machine learning describes a process through which the algorithm finds a set of characteristics that best fits the desired future outcome. In other words, it describes the scenario to determine correctly the classification results for patients who were not part of the training database. Here we were interested in knowing the characteristics under which a patient would continue treatment with < 10% chance of withdrawal. The results indicated that patients with ≥ 90% chance of continuing treatment fulfil the following criteria: (i) ≥ 23 years old at the time of diagnosis, (ii) ≤ 49 years old at the time of treatment, (iii) receiving ustekinumab rather than a tumour necrosis factor inhibitor, (iv) not diagnosed with psoriatic arthritis, (v) baseline DLQI ≥ 16, (vi) baseline PASI ≥ 9·4, (vii) no previous history of biologic failure and (viii) weight ≤ 98·9 kg. Potential limitations to this study include its retrospective nature and the utilization of a small sample dataset from a single country (five academic centres in Denmark). Moreover, the duration of treatment of biologics is not only dependent on the objective responses but may be heavily influenced by nonmedical factors, such as access to the medication, reimbursement or treatment guidelines. Thus, the accuracy of our algorithm may differ between centres, and each centre should establish their own algorithm to reflect local clinical practices. Further studies and analyses on a larger, recent and more diverse dataset are needed to identify universally applicable criteria for drug discontinuation. Funding sources: this study was funded by Alberta Innovates and the Canadian Association of Psoriasis Patients (to A.X.D.) and a start‐up grant from the Department of Medicine, University of Alberta (to R. Gniadecki). Conflicts of interest: S.E., A.X.D., P.S. and R. Greiner report no conflicts. S.F.T. has been a paid speaker for AbbVie, Eli Lilly, Novartis, Sanofi, Pierre Fabre, GSK and LEO Pharma; and has served on advisory boards with AbbVie, Eli Lilly, Janssen, Novartis, Roche, Sanofi, UCB and LEO Pharma. He has served as an investigator for AbbVie, AstraZeneca, Boehringer, UCB, CSL and Novartis and received research grants from AbbVie, Novartis, Sanofi and UCB. R. Gniadecki reports carrying out clinical trials for AbbVie and Janssen and has received honoraria as a consultant and/or speaker from AbbVie, Bausch Health, Eli Lilly, Janssen, Mallinckrodt, Novartis and Sanofi. The authors do not have equity in pharmaceutical companies. The full version of this paper has been published as a preprint on arXiv (arXiv:1908·09251).