On Semi-supervised Estimation of Discrete Distributions under f-divergences

We study the problem of estimating the joint probability mass function (pmf) over two random variables. In particular, the estimation is based on the observation of m samples containing both variables and n samples missing one fixed variable. We adopt the minimax framework with lᵖₚ loss functions. Recent work established that univariate minimax estimator combinations achieve minimax risk with the optimal first-order constant for p 2 in the regime m = o (n), questions remained for p 2 and various f-divergences. In our study, we affirm that these composite estimators are indeed minimax optimal for lᵖₚ loss functions, specifically for the range 1 p 2, including the critical l₁ loss. Additionally, we ascertain their optimality for a suite of f-divergences, such as KL, ², Squared Hellinger, and Le Cam divergences.