Nonparametric Approximate Dynamic Programming via the Kernel Method

This paper presents a novel, non-parametric approximate dynamic programming (ADP) algorithm that enjoys dimension-independent approximation and sample complexity guarantees. We obtain this algorithm by “kernelizing” a recent mathematical program for ADP (the “smoothed approximate linear program”). Loosely, our guarantees show that we can exchange the importance of choosing a good approximation architecture a priori (as required by existing approaches) with sampling effort. We also present a simple active set algorithm for solving the resulting quadratic program, and prove the correctness of this method. Via a computational study on a controlled queueing network, we show that our approach is capable of outperforming parametric linear programming approaches to ADP, as well as non-trivial, tailored heuristics for the same network, even when employing generic, polynomial kernels.