Towards a Better Understanding of Predict and Count Models
원문 초록
In a recent paper, Levy and Goldberg pointed out an interesting connection between prediction-based word embedding models and count models based on pointwise mutual information. Under certain conditions, they showed that both models end up optimizing equivalent objective functions. This paper explores this connection in more detail and lays out the factors leading to differences between these models. We find that the most relevant differences from an optimization perspective are (i) predict models work in a low dimensional space where embedding vectors can interact heavily; (ii) since predict models have fewer parameters, they are less prone to overfitting. Motivated by the insight of our analysis, we show how count models can be regularized in a principled manner and provide closed-form solutions for L1 and L2 regularization. Finally, we propose a new embedding model with a convex objective and the additional benefit of being intelligible.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
DACN: Dual-Attention Convolutional Network for Hyperspectral Image Super-Resolution
WALINET: A water and lipid identification convolutional Neural Network for nuisance signal removal in 1H MR Spectroscopic Imaging
Monkeypox disease recognition model based on improved SE-InceptionV3
Convergence of a L2 regularized Policy Gradient Algorithm for the Multi Armed Bandit
Prevalidated ridge regression is a highly-efficient drop-in replacement for logistic regression for high-dimensional data
Gradient-based bilevel optimization for multi-penalty Ridge regression through matrix differential calculus
토론
이 논문과 연결된 포럼 글이 아직 없습니다.