Papers with Code 아카이브
Audio captioning
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 2
- 평가표
- 2
- 보존된 평가 기록
- 29
Audio Captioning is the task of describing audio using text. The general approach is to use an audio encoder to encode the audio (example: PANN, CAV-MAE), and to use a decoder (example: transformer) to generate the text. To judge the quality of audio captions, though machine translation metrics (BLEU, METEOR, ROUGE) and image captioning metrics (SPICE, CIDER) are used, they are not very well-suited. Attempts have been made to use pretrained language model based metrics such as Sentence-BERT.
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
AC/DC: LLM-based Audio Comprehension via Dialogue Continuation
CLAP-ART: Automated Audio Captioning with Semantic-rich Audio Representation Tokenizer
FusionAudio-1.2M: Towards Fine-grained Audio Captioning with Multimodal Contextual Fusion
Mitigating Audiovisual Mismatch in Visual-Guide Audio Captioning
TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretraining
M2D2: Exploring General-purpose Audio-Language Representations Beyond CLAP
Solla: Towards a Speech-Oriented LLM That Hears Acoustic Context
Mellow: a small audio language model for reasoning
Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities
Enhancing Speech Large Language Models with Prompt-Aware Mixture of Audio Encoders
Unbiased Sliced Wasserstein Kernels for High-Quality Audio Captioning
ADIFF: Explaining audio difference using natural language
LLMs can see and hear without any training
CosyAudio: Improving Audio Generation with Confidence Scores and Synthetic Captions
LAVCap: LLM-based Audio-Visual Captioning using Optimal Transport
Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Model
Classifier-Guided Captioning Across Modalities
ETTA: Elucidating the Design Space of Text-to-Audio Models
AudioSetCaps: An Enriched Audio-Caption Dataset using Automated Generation Pipeline with Large Audio and Language Models
Tell What You Hear From What You See -- Video to Audio Generation Through Text
EmotionCaps: Enhancing Audio Captioning Through Emotion-Augmented Data Generation
Enhancing Retrieval-Augmented Audio Captioning with Generation-Assisted Multimodal Querying and Progressive Learning
DRCap: Decoding CLAP Latents with Retrieval-Augmented Generation for Zero-shot Audio Captioning
SLAM-AAC: Enhancing Audio Captioning with Paraphrasing Augmentation and CLAP-Refine through LLMs
Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization
An Eye for an Ear: Zero-shot Audio Description Leveraging an Image Captioner using Audiovisual Distribution Alignment
OpenSep: Leveraging Large Language Models with Textual Inversion for Open World Audio Separation
CLAIR-A: Leveraging Large Language Models to Judge Audio Captions
Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models
Towards Diverse and Efficient Audio Captioning via Diffusion Models
Enhancing Temporal Understanding in Audio Question Answering for Large Audio Language Models
Expanding on EnCLAP with Auxiliary Retrieval Model for Automated Audio Captioning
EnCLAP++: Analyzing the EnCLAP Framework for Optimizing Automated Audio Captioning Performance
Taming Data and Transformers for Audio Generation
Enhancing Automated Audio Captioning via Large Language Models with Optimized Audio Encoding
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.