Online Learning Methods for Networking

In this monograph we provide a tutorial on a family of sequential learning and decision problems known as the multi-armed bandit problems. We introduce a wide range of application scenarios for this learning framework, as well as its many different variants. The more detailed discussion is focused on the stochastic bandit problems, with rewards driven by either an IID or a Markov process, and when the environment consists of a single or multiple simultaneous users. We also present literature on the learning of MDPs, which captures coupling among the evolution of different options that a classical MAB problem does not.