New Projection-free Algorithms for Online Convex Optimization with Adaptive Regret Guarantees

We present new efficient -free algorithms for online optimization (OCO), where by projection-free we refer to algorithms that computing orthogonal projections onto the feasible set, and instead relay different and potentially much more efficient oracles. While most-of-the-art projection-free algorithms are based on the-the-leader framework, our algorithms are fundamentally and are based on the gradient descent algorithm with novel and efficient approach to computing so-called . As a consequence, we obtain the first projection-free algorithms naturally yield regret guarantees, i.e., regret bounds hold w.r.t. any sub-interval of the sequence. Concretely, when assuming availability of a linear optimization oracle (LOO) for the feasible set, on sequence of length T, our algorithms guarantee O(T3/4) adaptive regret O(T3/4) adaptive expected regret, for the full-information and bandit, respectively, using only $O(T)$ calls to the LOO. These bounds match current state-of-the-art regret bounds for LOO-based projection-free OCO, are adaptive. We also consider a new natural setting in the feasible set is accessible through a separation oracle. We present which, using overall $O(T)$ calls to the separation oracle, O(\√T) adaptive regret and O(T3/4) adaptive expected for the full-information and bandit settings, respectively.