A Reverse Jensen Inequality Result with Application to Mutual Information Estimation
원문 초록
The Jensen inequality is a widely used tool in a multitude of fields, such as for example information theory and machine learning. It can be also used to derive other standard inequalities such as the inequality of arithmetic and geometric means or the H\"older inequality. In a probabilistic setting, the Jensen inequality describes the relationship between a convex function and the expected value. In this work, we want to look at the probabilistic setting from the reverse direction of the inequality. We show that under minimal constraints and with a proper scaling, the Jensen inequality can be reversed. We believe that the resulting tool can be helpful for many applications and provide a variational estimation of mutual information, where the reverse inequality leads to a new estimator with superior training behavior compared to current estimators.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Closed-Form Information Capacity of Canonical Signaling Models
Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
On the Properties and Estimation of Pointwise Mutual Information Profiles
MINDE: Mutual Information Neural Diffusion Estimation
Rethinking Negative Pairs in Code Search
토론
이 논문과 연결된 포럼 글이 아직 없습니다.