Omnichannel Supply Chains Amid Demand Shocks: A Centralized Hierarchical Reinforcement Learning Framework

Background: The rapid evolution of omnichannel retailing has reshaped retail supply chains (SCs) by coupling replenishment, fulfillment, and service decisions across multiple demand channels under inventory, lead-time, and capacity constraints. These interdependencies create coordination challenges, particularly when demand shocks interact with limited operational capacity. Methods: To address these challenges, this study develops a centralized Hierarchical Reinforcement Learning (HRL) control framework that makes decision timing explicit: replenishment and allocation are optimized weekly, while fulfillment and lateral inventory rebalancing are controlled daily. Policies are learned using Proximal Policy Optimization (PPO) in an actor–critic architecture, with bounded stochastic policies for constrained action spaces. To mitigate the curse of dimensionality in HRL, we introduce a capacity-aware state–action encoding mechanism that compresses the control interface into structured summary signals. Demand shocks are modeled using two specifications: a mixed profile, where half the products follow a uniform demand process and the rest a Merton-type jump-diffusion process, and a fully shock-driven profile. Results: The framework is evaluated against forecast-driven base-stock and greedy fulfillment heuristics, and a perfect-information oracle, with pairwise differences examined through Wilcoxon signed-rank tests. Conclusions: Overall, the proposed framework improves learning efficiency and scalability, outperforming heuristic baselines while remaining below the oracle bound.