Unified Perspective on Probability Divergence via Maximum Likelihood Density Ratio Estimation: Bridging KL-Divergence and Integral Probability Metrics
원문 초록
This paper provides a unified perspective for the Kullback-Leibler (KL)-divergence and the integral probability metrics (IPMs) from the perspective of maximum likelihood density-ratio estimation (DRE). Both the KL-divergence and the IPMs are widely used in various fields in applications such as generative modeling. However, a unified understanding of these concepts has still been unexplored. In this paper, we show that the KL-divergence and the IPMs can be represented as maximal likelihoods differing only by sampling schemes, and use this result to derive a unified form of the IPMs and a relaxed estimation method. To develop the estimation problem, we construct an unconstrained maximum likelihood estimator to perform DRE with a stratified sampling scheme. We further propose a novel class of probability divergences, called the Density Ratio Metrics (DRMs), that interpolates the KL-divergence and the IPMs. In addition to these findings, we also introduce some applications of the DRMs, such as DRE and generative adversarial networks. In experiments, we validate the effectiveness of our proposed methods.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Savage-Dickey density ratio estimation with normalizing flows for Bayesian model comparison
Dequantified Diffusion-Schr{ö}dinger Bridge for Density Ratio Estimation
Learning the Optimal Stopping for Early Classification within Finite Horizons via Sequential Probability Ratio Test
Your copula is a classifier in disguise: classification-based copula density estimation
How to Leverage Demonstration Data in Alignment for Large Language Model? A Self-Imitation Learning Perspective
Robust Conformal Volume Estimation in 3D Medical Images
토론
이 논문과 연결된 포럼 글이 아직 없습니다.