We study stochastic gradient descent (SGD) with gradient clipping on convex functions under a generalized smoothness assumption called (L₀, L₁) -smoothness. Using gradient clipping, we establish a high probability convergence rate that matches the SGD rate in the L smooth case up to polylogarithmic factors and additive terms. We also propose a variation of adaptive SGD with gradient clipping, which achieves the same guarantee. We perform empirical experiments to examine our theory and algorithmic choices.
Gaash et al. (Sun,) studied this question.