https://github.com/DK-Nguyen/audio-captioning-sub-sampling
공식 코드Temporal Sub-sampling of Audio Feature Sequences for Automated Audio Captioning
Papers with Code 아카이브
Audio captioning
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 2
- 평가표
- 2
- 보존된 평가 기록
- 29
Audio Captioning is the task of describing audio using text. The general approach is to use an audio encoder to encode the audio (example: PANN, CAV-MAE), and to use a decoder (example: transformer) to generate the text. To judge the quality of audio captions, though machine translation metrics (BLEU, METEOR, ROUGE) and image captioning metrics (SPICE, CIDER) are used, they are not very well-suited. Attempts have been made to use pretrained language model based metrics such as Sentence-BERT.
코드
관련 논문과 평가 결과에 연결된 구현입니다. 출처에서 확인된 저자 구현에는 공식 코드 표시가 있습니다.
https://github.com/DragonLiu1995/multimodal-llm-for-audio-gen
공식 코드Tell What You Hear From What You See -- Video to Audio Generation Through Text
https://github.com/JanBerg1/AAC-LwF
공식 코드Continual Learning for Automated Audio Captioning Using The Learning Without Forgetting Approach
https://github.com/MinkyuKim26/Prefix_AAC_ICASSP2023
공식 코드https://github.com/NVIDIA/audio-flamingo
공식 코드Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities
https://github.com/TXH-mercury/VALOR
공식 코드VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
https://github.com/X-LANCE/SLAM-LLM
공식 코드DRCap: Decoding CLAP Latents with Retrieval-Augmented Generation for Zero-shot Audio Captioning
https://github.com/XinhaoMei/ACT
공식 코드https://github.com/XinhaoMei/DCASE2021_task6_v2
공식 코드An Encoder-Decoder Based Audio Captioning System With Transfer and Reinforcement Learning
https://github.com/akoepke/audio-retrieval-benchmark
공식 코드Audio Retrieval with Natural Language Queries: A Benchmark Study
https://github.com/ali-vosoughi/counterfactual-audio
공식 코드Learning Audio Concepts from Counterfactual Natural Language
https://github.com/amphionspace/SA-Eval
공식 코드Solla: Towards a Speech-Oriented LLM That Hears Acoustic Context
https://github.com/blmoistawinde/fense
공식 코드https://github.com/bytedance/salmonn
공식 코드SALMONN: Towards Generic Hearing Abilities for Large Language Models
https://github.com/bytedance/video-salmonn-2
공식 코드video-SALMONN 2: Captioning-Enhanced Audio-Visual Large Language Models
https://github.com/davidmchan/clair-a
공식 코드CLAIR-A: Leveraging Large Language Models to Judge Audio Captions
https://github.com/emrcak/dcase-2020-baseline
공식 코드Multi-task Regularization Based on Infrequent Classes for Audio Captioning
https://github.com/explainableml/zeraucap
공식 코드Zero-shot audio captioning with audio-language model guidance and audio context keywords
https://github.com/facebookresearch/mils
공식 코드https://github.com/frankenliu/LOAE
공식 코드Enhancing Automated Audio Captioning via Large Language Models with Optimized Audio Encoding
https://github.com/haantran96/wavetransformer
공식 코드https://github.com/hugomalard/aneyeforanear
공식 코드https://github.com/ilaria-manco/muscaps
공식 코드https://github.com/jaeyeonkim99/enclap
공식 코드EnCLAP: Combining Neural Audio Codec and Audio-Text Joint Embedding for Automated Audio Captioning
https://github.com/jishengbai/audiolog
공식 코드AudioLog: LLMs-Powered Long Audio Logging with Hybrid Token-Semantic Contrastive Learning
https://github.com/jishengbai/audiosetcaps
공식 코드https://github.com/jiwanchung/esper
공식 코드https://github.com/kuan2jiu99/audio-hallucination
공식 코드https://github.com/labbeti/aac-metrics
공식 코드https://github.com/littleflyingsheep/p-localaft
공식 코드Local Information Assisted Attention-free Decoder for Audio Captioning
https://github.com/liuxubo717/cl4ac
공식 코드https://github.com/liuxubo717/contrastive_loss_for_audio_captioning
공식 코드https://github.com/liuxubo717/v-act
공식 코드Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention
https://github.com/microsoft/noaudiocaptioning
공식 코드https://github.com/microsoft/pengi
공식 코드https://github.com/microsoft/wavtext5k
공식 코드출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.