Audio Captioning using Pre-Trained Large-Scale Language Model Guided by Audio-based Similar Caption Retrieval
원문 초록
The goal of audio captioning is to translate input audio into its description using natural language. One of the problems in audio captioning is the lack of training data due to the difficulty in collecting audio-caption pairs by crawling the web. In this study, to overcome this problem, we propose to use a pre-trained large-scale language model. Since an audio input cannot be directly inputted into such a language model, we utilize guidance captions retrieved from a training dataset based on similarities that may exist in different audio. Then, the caption of the audio input is generated by using a pre-trained language model while referring to the guidance captions. Experimental results show that (i) the proposed method has succeeded to use a pre-trained language model for audio captioning, and (ii) the oracle performance of the pre-trained model-based caption generator was clearly better than that of the conventional method trained from scratch.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Kodezi Chronos: A Debugging-First Language Model for Repository-Scale, Memory-Driven Code Understanding
Differential Mamba
RMIT-ADM+S at the SIGIR 2025 LiveRAG Challenge
FlexRAG: A Flexible and Comprehensive Framework for Retrieval-Augmented Generation
XGraphRAG: Interactive Visual Analysis for Graph-based Retrieval-Augmented Generation
Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Model
토론
이 논문과 연결된 포럼 글이 아직 없습니다.