Show from Tell: Audio-Visual Modelling in Clinical Settings
원문 초록
Auditory and visual signals usually present together and correlate with each other, not only in natural environments but also in clinical settings. However, the audio-visual modelling in the latter case can be more challenging, due to the different sources of audio/video signals and the noise (both signal-level and semantic-level) in auditory signals -- usually speech. In this paper, we consider audio-visual modelling in a clinical setting, providing a solution to learn medical representations that benefit various clinical tasks, without human expert annotation. A simple yet effective multi-modal self-supervised learning framework is proposed for this purpose. The proposed approach is able to localise anatomical regions of interest during ultrasound imaging, with only speech audio as a reference. Experimental evaluations on a large-scale clinical multi-modal ultrasound video dataset show that the proposed self-supervised method learns good transferable anatomical representations that boost the performance of automated downstream clinical tasks, even outperforming fully-supervised solutions.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
RetFiner: A Vision-Language Refinement Scheme for Retinal Foundation Models
Boosting Generative Adversarial Transferability with Self-supervised Vision Transformer Features
Post-training for Deepfake Speech Detection
Topology-Aware Modeling for Unsupervised Simulation-to-Reality Point Cloud Recognition
Efficient and Generalizable Speaker Diarization via Structured Pruning of Self-Supervised Models
Pushing the Performance of Synthetic Speech Detection with Kolmogorov-Arnold Networks and Self-Supervised Learning Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.