Lipreading
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 1
- 평가표
- 8
- 보존된 평가 기록
- 92
Lipreading is a process of extracting speech by watching lip movements of a speaker in the absence of sound. Humans lipread all the time without even noticing. It is a big part in communication albeit not as dominant as audio. It is a very helpful skill to learn especially for those who are hard of hearing.
Deep Lipreading is the process of extracting speech from a video of a silent talking face using deep neural networks. It is also known by few other names: Visual Speech Recognition (VSR), Machine Lipreading, Automatic Lipreading etc.
The primary methodology involves two stages: i) Extracting visual and temporal features from a sequence of image frames from a silent talking video ii) Processing the sequence of features into units of speech e.g. characters, words, phrases etc. We can find several implementations of this methodology either done in two separate stages or trained end-to-end in one go.
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
Learning Contextually Fused Audio-visual Representations for Audio-visual Speech Recognition
Robust Self-Supervised Audio-Visual Speech Recognition
Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction
Personalized One-Shot Lipreading for an ALS Patient
Sub-word Level Lip Reading With Visual Attention
LRWR: Large-Scale Benchmark for Lip Reading in Russian language
Large-vocabulary Audio-visual Speech Recognition in Noisy Environments
Adaptive Semantic-Spatio-Temporal Graph Convolutional Network for Lip Reading
End-to-end Audio-visual Speech Recognition with Conformers
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
Part-based Lipreading for Audio-Visual Speech Recognition
Learn an Effective Lip Reading Model without Pains
FastLR: Non-Autoregressive Lipreading Model with Integrate-and-Fire
Towards Practical Lipreading with Distilled and Efficient Models
SpotFast Networks with Memory Augmented Lateral Transformers for Lipreading
Audio-visual Multi-channel Recognition of Overlapped Speech
Discriminative Multi-modality Speech Recognition
Mutual Information Maximization for Effective Lip Reading
Deformation Flow Based Two-Stream Network for Lip Reading
Pseudo-Convolutional Policy Gradient for Sequence-to-Sequence Lip-Reading
Can We Read Speech Beyond the Lips? Rethinking RoI Selection for Deep Visual Speech Recognition
Lipreading using Temporal Convolutional Networks
Audio-visual Recognition of Overlapped speech for the LRS2 dataset
ASR is all you need: cross-modal distillation for lip reading
Hearing Lips: Improving Lip Reading by Distilling Speech Recognizers
Recurrent Neural Network Transducer for Audio-Visual Speech Recognition
Alternative Visual Units for an Optimized Phoneme-Based Lipreading System
Multi-Grained Spatio-temporal Modeling for Lip-reading
A Cascade Sequence-to-Sequence Model for Chinese Mandarin Lip Reading
Lipper: Synthesizing Thy Speech using Multi-View Lipreading
LipReading with 3D-2D-CNN BLSTM-HMM and word-CTC models
Learning Spatio-Temporal Features with Two-Stream Deep 3D CNNs for Lipreading
Learning from Videos with Deep Convolutional LSTM Networks
On the Importance of Video Action Recognition for Visual Lipreading
Pushing the boundaries of audiovisual word recognition using Residual Networks and LSTMs
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.