Nonparanormal Information Estimation
원문 초록
We study the problem of using i.i.d. samples from an unknown multivariate probability distribution $p$ to estimate the mutual information of $p$. This problem has recently received attention in two settings: (1) where $p$ is assumed to be Gaussian and (2) where $p$ is assumed only to lie in a large nonparametric smoothness class. Estimators proposed for the Gaussian case converge in high dimensions when the Gaussian assumption holds, but are brittle, failing dramatically when $p$ is not Gaussian. Estimators proposed for the nonparametric case fail to converge with realistic sample sizes except in very low dimensions. As a result, there is a lack of robust mutual information estimators for many realistic data. To address this, we propose estimators for mutual information when $p$ is assumed to be a nonparanormal (a.k.a., Gaussian copula) model, a semiparametric compromise between Gaussian and nonparametric extremes. Using theoretical bounds and experiments, we show these estimators strike a practical balance between robustness and scaling with dimensionality.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Closed-Form Information Capacity of Canonical Signaling Models
Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
On the Properties and Estimation of Pointwise Mutual Information Profiles
MINDE: Mutual Information Neural Diffusion Estimation
Rethinking Negative Pairs in Code Search
토론
이 논문과 연결된 포럼 글이 아직 없습니다.