This is the authors' abstract. We don't add key points for this paper.
Flow-based generative models enjoy certain advantages in computing the data generation and the likelihood, and have recently shown competitive empirical performance. Compared to the accumulating theoretical studies on related score-based diffusion models, analysis of flow-based models, which are deterministic in both forward (data-to-noise) and reverse (noise-to-data) directions, remain sparse. In this paper, we provide a theoretical guarantee of generating data distribution by a progressive flow model, the so-called JKO flow model, which implements the Jordan-Kinderleherer-Otto (JKO) scheme in a normalizing flow network. Leveraging the exponential convergence of the proximal gradient descent (GD) in Wasserstein space, we prove the Kullback-Leibler (KL) guarantee of data generation by a JKO flow model to be <tex-math notation="LaTeX">O(ε ²) </tex-math> when using <tex-math notation="LaTeX">N log (1/ε) </tex-math> many JKO steps (N Residual Blocks in the flow) where <tex-math notation="LaTeX">ε </tex-math> is the error in the per-step first-order condition. The assumption on data density is merely a finite second moment, and the theory extends to data distributions without density and when there are inversion errors in the reverse process where we obtain KL- <tex-math notation="LaTeX">W₂ </tex-math> mixed error guarantees. The non-asymptotic convergence rate of the JKO-type <tex-math notation="LaTeX">W₂ </tex-math>-proximal GD is proved for a general class of convex objective functionals that includes the KL divergence as a special case, which can be of independent interest. The analysis framework can extend to other first-order Wasserstein optimization schemes applied to flow-based generative models.
Answers are grounded in this paper and cite their sources.
No takes yet. Share an insight, caveat, or question.
Cheng et al. (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: