Reinforcement learning evaluation of treatment policies for patients with hepatitis C virus

Figures

• 

  Fig. 1. Modeling approach for reinforcement learning and off-policy evaluation. The historical cohort dataset consists of patients (1), whose state, i.e., longitudinal and demographic information is measured (2). Given these measurements, the risk to the patient progressing to cirrhosis is then evaluated (3). Finally, following the usual care treatment policy (4), a clinician makes a treatment decision (5) for the patient. The cycle then continues until the patient no longer returns for follow-up or the follow-up period concludes

• 

  Fig. 2. Treatment probabilities as a function of APRI score for the (a) piecewise treatment policy and (b) logistic regression (APRI only) policy. The first treatment policy is a piecewise function where treatment probability increases with APRI score (a). The second treatment policy is a data-driven treatment policy using logistic regression with APRI as a single feature, and the outcome being a positive diagnosis of cirrhosis (b)

• 

  Fig. 3. Three example traces for patients. Treatment decisions over time (a), SVR status (b), and risk score for development of cirrhosis and APRI/300, where 300 was chosen to place risk and APRI on similar scales (c) are displayed

• 

  Fig. 4. Analysis of risk scores. a Comparison of risk scores in the dataset separated between treated and untreated measurement timepoints. As expected, the untreated timepoints have a higher risk score on average. b Median risk (with 50% percentile interval) striated across amount of time after treatment start date. As expected, risk continues to decrease after initial treatment

• 

  Fig. 5. Risk sensitivity for k and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${r}_{0}$$\end{document}r0 . Sensitivity of the risk-based policy to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{r}}_{0}$$\end{document}r0 and\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$k$$\end{document}k. As \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${r}_{0}$$\end{document}r0 increases, risk also increases for a fixed \documentclass[12pt]{minimal} \usepackage{amsmath}…