From Heuristics to Reinforcement Learning: Integrated Operational–Financial Control of Supply Chains Under Demand Disruption

Supply chain control requires balancing operational performance and financial efficiency when decisions are made using delayed and imperfect demand information. Although fixed heuristics, adaptive policies, and reinforcement learning approaches have been proposed, their relative effectiveness and robustness under temporary informational mismatch remain unclear. This study addresses this gap by developing an integrated simulation–reinforcement learning framework that jointly captures operational and financial dynamics in supply chains, which enables adaptive optimisation of working capital policies under uncertainty. A unified simulation framework is developed for a multi-echelon supply chain that jointly models service levels, backlog, customer retention, and working capital exposure through the cash conversion cycle. Five classes of controllers are evaluated: fixed-threshold heuristics, adaptive threshold policies optimised using stochastic and evolutionary search, and a reinforcement learning controller based on proximal policy optimisation. Performance is assessed under stationary demand and under demand disruptions. The results reveal a clear hierarchy of performance. Fixed heuristics provide transparent and stable baselines but suffer from structural rigidity. Adaptive threshold policies substantially improve coordination, with evolutionary search yielding the strongest performance among structured approaches. The reinforcement learning controller achieves the best overall outcomes by learning a nonlinear state–action mapping that sharply reduces backlog and service shortfalls while maintaining comparable working capital exposure. These gains arise from improved coordination across operational and financial decisions rather than single-metric optimisation. Practically, adaptive heuristics offer robust baselines, while learning-based controllers are most valuable in more volatile environments.