Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Space
원문 초록
This paper explores image caption generation using conditional variational auto-encoders (CVAEs). Standard CVAEs with a fixed Gaussian prior yield descriptions with too little variability. Instead, we propose two models that explicitly structure the latent space around $K$ components corresponding to different types of image content, and combine components to create priors for images that contain multiple types of content simultaneously (e.g., several kinds of objects). Our first model uses a Gaussian Mixture model (GMM) prior, while the second one defines a novel Additive Gaussian (AG) prior that linearly combines component means. We show that both models produce captions that are more diverse and more accurate than a strong LSTM baseline or a "vanilla" CVAE with a fixed Gaussian prior, with AG-CVAE showing particular promise.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
DiffCap: Exploring Continuous Diffusion on Image Captioning
Text-Visual Semantic Constrained AI-Generated Image Quality Assessment
DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World
SonicVerse: Multi-Task Learning for Music Feature-Informed Captioning
FusionAudio-1.2M: Towards Fine-grained Audio Captioning with Multimodal Contextual Fusion
VCapsBench: A Large-scale Fine-grained Benchmark for Video Caption Quality Evaluation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.