Automated Audio Captioning with Recurrent Neural Networks
원문 초록
We present the first approach to automated audio captioning. We employ an encoder-decoder scheme with an alignment model in between. The input to the encoder is a sequence of log mel-band energies calculated from an audio file, while the output is a sequence of words, i.e. a caption. The encoder is a multi-layered, bi-directional gated recurrent unit (GRU) and the decoder a multi-layered GRU with a classification layer connected to the last GRU of the decoder. The classification layer and the alignment model are fully connected layers with shared weights between timesteps. The proposed method is evaluated using data drawn from a commercial sound effects library, ProSound Effects. The resulting captions were rated through metrics utilized in machine translation and image captioning fields. Results from metrics show that the proposed method can predict words appearing in the original caption, but not always correctly ordered.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Document-Level Text Generation with Minimum Bayes Risk Decoding using Optimal Transport
A Novel Lightweight Transformer with Edge-Aware Fusion for Remote Sensing Image Captioning
Representation Learning through Multimodal Attention and Time-Sync Comments for Affective Video Content Analysis
Effective Deep Learning Models for Automatic Diacritization of Arabic Text
Beyond the Attention: Distinguish the Discriminative and Confusable Features For Fine-grained Image Classification
Hierarchical memory decoder for visual narrating
토론
이 논문과 연결된 포럼 글이 아직 없습니다.