Reinforcement learning optimization of automated insulin delivery in type 1 and type 2 diabetes mellitus

Closed-loop insulin delivery systems have proven effective in regulating blood glucose (BG) concentration, thereby reducing the burden of self-care in the management of type 1 and type 2 diabetes mellitus. However, the prevalence of unexpected disturbances resulting from oral glucose intake represents a considerable challenge to the full automation of these systems. Here, we propose an actor-critic reinforcement learning (RL) framework implemented within environments governed by compartmental ordinary differential equation models of the glucose–insulin–glucagon–incretins dynamics. This approach was employed to optimize automated insulin delivery in virtual patients with type 1 and type 2 diabetes mellitus, under scenarios involving unforeseen BG disturbances. The resulting optimal RL policies were tested in silico on virtual patients subjected to three unannounced glucose disturbances over the course of a day. The findings demonstrated that optimal RL policies could sustain the BG a significantly higher percentage of time within the normoglycemic range and a significantly lower percentage of time below the normoglycemic range in comparison to either continuous or discrete proportional-integral-derivative control algorithms. These results set the basis for developing new approaches to optimizing automated dosing regimens for chronic disease management.