Bandit-Based Rate Adaptation for a Single-Server Queue

This paper considers the problem of obtaining bounded time-average expected queue sizes in a single-queue system with a partial-feedback structure. Time is slotted; in slot t the transmitter chooses a rate $V(t)$ from a continuous interval. Transmission succeeds if and only if V(t)≤ C(t), where channel capacities (t)\ and arrivals are i.i.d. draws from fixed but unknown distributions. The transmitter observes only binary acknowledgments (ACK/NACK) indicating success or failure. Let ε>0 denote a sufficiently small lower bound on the slack between the arrival rate and the capacity region. We propose a phased algorithm that progressively refines a discretization of the uncountable infinite rate space and, without knowledge of ε, achieves a O\!(log3.5(1/ε)/ε³) time-average expected queue size uniformly over the horizon. We also prove a converse result showing that for any rate-selection algorithm, regardless of whether ε is known, there exists an environment in which the worst-case time-average expected queue size is Ω(1/ε²). Thus, while a gap remains in the setting without knowledge of ε, we show that if ε is known, a simple single-stage UCB type policy with a fixed discretization of the rate space achieves O\!(log(1/ε)/ε²), matching the converse up to logarithmic factors.