Switching Costs and the Gittins Index

The Theorem of Gittins and Jones (1974) is, perhaps, the single most powerful result the literature on Bandit problems. This result establishes that in independent-armed problems with geometric discounting over an infinite horizon, all optimal strategies be obtained by solving a family of simple optimal stopping problems that with each arm an index known as the dynamic allocation index or, more , as the Gittins index. Importantly, the Gittins index of an arm depends solely the characteristics of that arm and the rate of discounting, and is otherwise independent of the problem under consideration. These features simplify the task of characterizing optimal strategies in this class of problems.