Intelligent Pricing and Ordering Decisions: A Deep Multi-agent Reinforcement Learning Algorithm

The dynamic pricing mechanisms of firms and search techniques for historical prices have spurred strategic behaviors and price expectations among customers. These customers predict future markdowns and delay their purchases based on price expectations (i.e., reference prices), where joint pricing and ordering decisions should be considered carefully to counteract or soften the negative impact of customers’ strategic behaviors and their reference prices on the seller’s profitability. Moreover, relevant literature neglects general and important features in the actual market (e.g., nonzero price thresholds and randomness in the formation of reference prices). In this study, we establish a Markov game between the retailer and strategic customers over a multi-period horizon, and set the reference price as a three-regime linear piecewise model with loss and gain thresholds. Then, we propose an adaptive multi-agent deep deterministic policy gradient (MA2DDPG) algorithm containing the experience replay buffer separation mechanism and delayed actor updates. Experimental results with synthetic and real-world datasets reveal that our algorithm converges to optimal policies in terms of convergence and rationality, and that it vastly outperforms the benchmark algorithms. Moreover, valuable managerial insights are obtained through an extensive numerical analysis for practitioners, particularly when they must consider the complicated characteristics of heterogeneous customer populations, such as disappointment behaviors, price thresholds, and strategic customer proportions. Our results pave the way for future research aimed at using multi-agent reinforcement learning to improve operations management with behavioral factors.