Visually grounded cross-lingual keyword spotting in speech
원문 초록
Recent work considered how images paired with speech can be used as supervision for building speech systems when transcriptions are not available. We ask whether visual grounding can be used for cross-lingual keyword spotting: given a text keyword in one language, the task is to retrieve spoken utterances containing that keyword in another language. This could enable searching through speech in a low-resource language using text queries in a high-resource language. As a proof-of-concept, we use English speech with German queries: we use a German visual tagger to add keyword labels to each training image, and then train a neural network to map English speech to German keywords. Without seeing parallel speech-transcriptions or translations, the model achieves a precision at ten of 58%. We show that most erroneous retrievals contain equivalent or semantically relevant keywords; excluding these would improve P@10 to 91%.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
GTA1: GUI Test-time Scaling Agent
High-Resolution Visual Reasoning via Multi-Turn Grounding-Based Reinforcement Learning
DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World
DrishtiKon: Multi-Granular Visual Grounding for Text-Rich Document Images
Advances in Small-Footprint Keyword Spotting: A Comprehensive Review of Efficient Models and Algorithms
GLAP: General contrastive audio-text pretraining across domains and languages
토론
이 논문과 연결된 포럼 글이 아직 없습니다.