On Semi-supervised Estimation of Discrete Distributions under f-divergences

We study the problem of estimating the joint probability mass function (pmf) over two random variables. In particular, the estimation is based on the observation of m samples containing both variables and n samples missing one fixed variable. We adopt the minimax framework with lᵖₚ loss functions. Recent work established that univariate minimax estimator combinations achieve minimax risk with the optimal first-order constant for p ≥ 2 in the regime $m = o(n)$, questions remained for p ≤ 2 and various f-divergences. In our study, we affirm that these composite estimators are indeed minimax optimal for lᵖₚ loss functions, specifically for the range 1 ≤ p ≤ 2, including the critical l₁ loss. Additionally, we ascertain their optimality for a suite of f-divergences, such as KL, χ², Squared Hellinger, and Le Cam divergences.