Zero-Shot Sign Language Recognition: Can Textual Data Uncover Sign Languages?
원문 초록
We introduce the problem of zero-shot sign language recognition (ZSSLR), where the goal is to leverage models learned over the seen sign class examples to recognize the instances of unseen signs. To this end, we propose to utilize the readily available descriptions in sign language dictionaries as an intermediate-level semantic representation for knowledge transfer. We introduce a new benchmark dataset called ASL-Text that consists of 250 sign language classes and their accompanying textual descriptions. Compared to the ZSL datasets in other domains (such as object recognition), our dataset consists of limited number of training examples for a large number of classes, which imposes a significant challenge. We propose a framework that operates over the body and hand regions by means of 3D-CNNs, and models longer temporal relationships via bidirectional LSTMs. By leveraging the descriptive text embeddings along with these spatio-temporal representations within a zero-shot learning framework, we show that textual data can indeed be useful in uncovering sign languages. We anticipate that the introduced approach and the accompanying dataset will provide a basis for further exploration of this new zero-shot learning problem.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
SeqNet: Sequential Networks for One-Shot Traffic Sign Recognition With Transfer Learning
Privacy-Enhanced Zero-Shot Learning via Data-Free Knowledge Transfer
Enhanced dynamic sign language recognition using slowfast networks
Continuous Sign Language Recognition Through Cross-Modal Alignment of Video and Text Embeddings in a Joint-Latent Space
Attribute-Based Classification for Zero-Shot Visual Object Categorization
RaMen: Multi-Strategy Multi-Modal Learning for Bundle Construction
토론
이 논문과 연결된 포럼 글이 아직 없습니다.