Multimodal Deep Learning
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 1
- 평가표
- 1
- 보존된 평가 기록
- 1
Multimodal deep learning is a type of deep learning that combines information from multiple modalities, such as text, image, audio, and video, to make more accurate and comprehensive predictions. It involves training deep neural networks on data that includes multiple types of information and using the network to make predictions based on this combined data.
One of the key challenges in multimodal deep learning is how to effectively combine information from multiple modalities. This can be done using a variety of techniques, such as fusing the features extracted from each modality, or using attention mechanisms to weight the contribution of each modality based on its importance for the task at hand.
Multimodal deep learning has many applications, including image captioning, speech recognition, natural language processing, and autonomous vehicles. By combining information from multiple modalities, multimodal deep learning can improve the accuracy and robustness of models, enabling them to perform better in real-world scenarios where multiple types of information are present.
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
Language-Assisted Deep Learning for Autistic Behaviors Recognition
aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception
MultiCrossViT: Multimodal Vision Transformer for Schizophrenia Prediction using Structural MRI and Functional Network Connectivity Data
Bayesian Prompt Learning for Image-Language Model Generalization
LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Models
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
LAVIS: A Library for Language-Vision Intelligence
TMSS: An End-to-End Transformer-based Multimodal Network for Segmentation and Survival Prediction
Identification of Cognitive Workload during Surgical Tasks with Multimodal Deep Learning
Multi-Modal Experience Inspired AI Creation
Multimodal Attention-based Deep Learning for Alzheimer's Disease Diagnosis
Vision-Aided Frame-Capture-Based CSI Recomposition for WiFi Sensing: A Multimodal Approach
Detection of Propaganda Techniques in Visuo-Lingual Metaphor in Memes
DeepGraviLens: a Multi-Modal Architecture for Classifying Gravitational Lensing Data
Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation
Multi-objective optimization determines when, which and how to fuse deep networks: an application to predict COVID-19 outcomes
A Review on Methods and Applications in Multimodal Deep Learning
Emotion Based Hate Speech Detection using Multimodal Learning
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena
CMA-CLIP: Cross-Modality Attention CLIP for Image-Text Classification
Geometric Multimodal Deep Learning with Multi-Scaled Graph Wavelet Convolutional Network
Multimodal Approach for Metadata Extraction from German Scientific Publications
From Multimodal to Unimodal Attention in Transformers using Knowledge Distillation
DeepStroke: An Efficient Stroke Screening Framework for Emergency Rooms with Multimodal Adversarial Deep Learning
Contrastive Language-Image Pre-training for the Italian Language
Pan-Cancer Integrative Histology-Genomic Analysis via Interpretable Multimodal Deep Learning
Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions
Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis
A Multimodal Deep Learning Model for Cardiac Resynchronisation Therapy Response Prediction
Deep Learning for Technical Document Classification
Listen to Your Favorite Melodies with img2Mxml, Producing MusicXML from Sheet Music Image by Measure-based Multimodal Deep Learning-driven Assembly
DeepMMSA: A Novel Multimodal Deep Learning Method for Non-small Cell Lung Cancer Survival Analysis
Digital Taxonomist: Identifying Plant Species in Community Scientists' Photographs
How to select and use tools? : Active Perception of Target Objects Using Multimodal Deep Learning
Recent Advances and Trends in Multimodal Deep Learning: A Review
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.