Gradual proactive regulation of body state by reinforcement learning of homeostasis

Living systems maintain physiological variables such as temperature, blood pressure, and glucose within narrow ranges-a process known as homeostasis. Homeostasis involves not only reactive feedback but also anticipatory adjustments shaped by experience. Prior homeostatic reinforcement learning (HRL) models have provided a computational account of anticipatory regulation under homeostatic challenges. However, existing formulations lack mechanisms for gradual, trial-by-trial adjustment and for extinction learning. To address this issue, we developed a continuous HRL framework that enables trial-wise tuning of anticipatory regulation. The model incorporates biologically informed components: asymmetric reinforcement, weighting negative outcomes more than positive outcomes; and a dual-unit, context-gated inhibitory mechanism. We applied the framework to thermoregulatory conditioning with ethanol-induced hypothermia and successfully reproduced cue-triggered compensation, gradual tolerance, and rapid reacquisition after extinction. We then extended the framework to multiple physiological variables influenced by shared neural or hormonal control signals, where compensating one variable can necessarily incur costs in others (e.g., heating at the expense of a fuel-like resource). Under uneven regulatory priorities, deviations propagated through shared control, yielding cascading, system-wide failure to stabilize near the ideal state-a failure mode discussed in autonomic dysregulation (e.g., dysautonomia, myalgic encephalomyelitis/chronic fatigue syndrome). Overall, our framework provides a computational basis to advances a systems-level understanding of multi-organ homeostatic dysregulation in vivo.