Bandit Learning-Based Decision-Making for Networked Systems: Constraints, Switching, Delay, and Risk Aversion

Networked systems typically operate under limited resources, under strict stability and latency requirements, and with incomplete knowledge of their environments. The control of networked systems may therefore benefit greatly from learning-based decision-making in order to maximize overall network utility. However, this utility maximization goal is complicated by four notable challenges that recur across networked systems: constraints in the form of service guarantees and budgeted resource consumption, operational costs due to reconfiguration and service changes, i.e., switching costs, feedback delays that both shape user experience and influence what the learner observes, and risk aversion to changing system performance in user-centric systems. These four aspects are even more challenging when the performance-relevant random processes, e.g., job arrivals, service, user motion, and resource availability, are unknown.