An on-line learning algorithm for energy efficient delay constrained scheduling over a fading channel

In this paper, we consider the problem of energy efficient scheduling under average delay constraint for a single user fading channel. We propose a new approach for on-line implementation of the optimal packet scheduling algorithm. This approach is based on reformulating the value iteration equation by introducing a virtual state called post-decision state. The resultant value iteration equation becomes amenable to online implementation based on stochastic approximation. This approach has an advantage that an explicit knowledge of the probability distribution of the channel state as well as the arrivals is not required for the implementation. We prove that the on-line algorithm indeed converges to the optimal policy.