Empirical Analysis of Image Caption Generation using Deep Learning
원문 초록
Automated image captioning is one of the applications of Deep Learning which involves fusion of work done in computer vision and natural language processing, and it is typically performed using Encoder-Decoder architectures. In this project, we have implemented and experimented with various flavors of multi-modal image captioning networks where ResNet101, DenseNet121 and VGG19 based CNN Encoders and Attention based LSTM Decoders were explored. We have studied the effect of beam size and the use of pretrained word embeddings and compared them to baseline CNN encoder and RNN decoder architecture. The goal is to analyze the performance of each approach using various evaluation metrics including BLEU, CIDEr, ROUGE and METEOR. We have also explored model explainability using Visual Attention Maps (VAM) to highlight parts of the images which has maximum contribution for predicting each word of the generated caption.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
A Novel Lightweight Transformer with Edge-Aware Fusion for Remote Sensing Image Captioning
Attention-based transformer models for image captioning across languages: An in-depth survey and evaluation
Scaling Up Biomedical Vision-Language Models: Fine-Tuning, Instruction Tuning, and Multi-Modal Learning
MedBLIP: Fine-tuning BLIP for Medical Image Captioning
Representation Learning through Multimodal Attention and Time-Sync Comments for Affective Video Content Analysis
Global Object Proposals for Improving Multi-Sentence Video Descriptions
토론
이 논문과 연결된 포럼 글이 아직 없습니다.