Interactive decoding of words from visual speech recognition models
원문 초록
This work describes an interactive decoding method to improve the performance of visual speech recognition systems using user input to compensate for the inherent ambiguity of the task. Unlike most phoneme-to-word decoding pipelines, which produce phonemes and feed these through a finite state transducer, our method instead expands words in lockstep, facilitating the insertion of interaction points at each word position. Interaction points enable us to solicit input during decoding, allowing users to interactively direct the decoding process. We simulate the behavior of user input using an oracle to give an automated evaluation, and show promise for the use of this method for text input.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Transfer Learning from Visual Speech Recognition to Mouthing Recognition in German Sign Language
VisualSpeaker: Visually-Guided 3D Avatar Lip Synthesis
ViCocktail: Automated Multi-Modal Data Collection for Vietnamese Audio-Visual Speech Recognition
Cocktail-Party Audio-Visual Speech Recognition
CNVSRC 2024: The Second Chinese Continuous Visual Speech Recognition Challenge
Leveraging Large Language Models in Visual Speech Recognition: Model Scaling, Context-Aware Decoding, and Iterative Polishing
토론
이 논문과 연결된 포럼 글이 아직 없습니다.