Using Motion History Images with 3D Convolutional Networks in Isolated Sign Language Recognition
원문 초록
Sign language recognition using computational models is a challenging problem that requires simultaneous spatio-temporal modeling of the multiple sources, i.e. faces, hands, body, etc. In this paper, we propose an isolated sign language recognition model based on a model trained using Motion History Images (MHI) that are generated from RGB video frames. RGB-MHI images represent spatio-temporal summary of each sign video effectively in a single RGB image. We propose two different approaches using this RGB-MHI model. In the first approach, we use the RGB-MHI model as a motion-based spatial attention module integrated into a 3D-CNN architecture. In the second approach, we use RGB-MHI model features directly with the features of a 3D-CNN model using a late fusion technique. We perform extensive experiments on two recently released large-scale isolated sign language datasets, namely AUTSL and BosphorusSign22k. Our experiments show that our models, which use only RGB data, can compete with the state-of-the-art models in the literature that use multi-modal data.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Bukva: Russian Sign Language Alphabet
HandReader: Advanced Techniques for Efficient Fingerspelling Recognition
Hierarchical Sub-action Tree for Continuous Sign Language Recognition
SignBart -- New approach with the skeleton sequence for Isolated Sign language Recognition
SLRNet: A Real-Time LSTM-Based Sign Language Recognition System
Transfer Learning from Visual Speech Recognition to Mouthing Recognition in German Sign Language
토론
이 논문과 연결된 포럼 글이 아직 없습니다.