Papers with Code 아카이브
Audio captioning
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 2
- 평가표
- 2
- 보존된 평가 기록
- 29
Audio Captioning is the task of describing audio using text. The general approach is to use an audio encoder to encode the audio (example: PANN, CAV-MAE), and to use a decoder (example: transformer) to generate the text. To judge the quality of audio captions, though machine translation metrics (BLEU, METEOR, ROUGE) and image captioning metrics (SPICE, CIDER) are used, they are not very well-suited. Attempts have been made to use pretrained language model based metrics such as Sentence-BERT.
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
Improving Text-To-Audio Models with Synthetic Captions
Understanding Sounds, Missing the Questions: The Challenge of Object Hallucination in Large Audio-Language Models
Audio Dialogues: Dialogues dataset for audio and music understanding
Improved Baselines for Data-efficient Perceptual Augmentation of LLMs
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities
EnCLAP: Combining Neural Audio Codec and Audio-Text Joint Embedding for Automated Audio Captioning
Learning Audio Concepts from Counterfactual Natural Language
AudioLog: LLMs-Powered Long Audio Logging with Hybrid Token-Semantic Contrastive Learning
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
Zero-shot audio captioning with audio-language model guidance and audio context keywords
SALMONN: Towards Generic Hearing Abilities for Large Language Models
LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT
Weakly-supervised Automated Audio Captioning via text only training
Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning
RECAP: Retrieval-Augmented Audio Captioning
Audio Difference Learning for Audio Captioning
Training Audio Captioning Models without Audio
Zero-Shot Audio Captioning via Audibility Guidance
Parameter Efficient Audio Captioning With Faithful Guidance Using Audio-text Shared Latent Representation
Generating Realistic Images from In-the-wild Sounds
Killing two birds with one stone: Can an audio captioning system also be used for audio-text retrieval?
Audio Difference Captioning Utilizing Similarity-Discrepancy Disentanglement
Crowdsourcing and Evaluating Text-Based Audio Retrieval Relevances
Improving Audio Caption Fluency with Automatic Error Correction
Dual Transformer Decoder based Features Fusion Network for Automated Audio Captioning
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset
Pengi: An Audio Language Model for Audio Tasks
A Whisper transformer for audio captioning trained with synthetic captions and transfer learning
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
Efficient Audio Captioning Transformer with Patchout and Text Guidance
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research
Prefix tuning for automated audio captioning
Towards Generating Diverse Audio Captions via Adversarial Training
Impact of visual assistance for automated audio captioning
Is my automatic audio captioning system so bad? spider-max: a metric to consider several caption candidates
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.