Multimodal Deep Learning
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 1
- 평가표
- 1
- 보존된 평가 기록
- 1
Multimodal deep learning is a type of deep learning that combines information from multiple modalities, such as text, image, audio, and video, to make more accurate and comprehensive predictions. It involves training deep neural networks on data that includes multiple types of information and using the network to make predictions based on this combined data.
One of the key challenges in multimodal deep learning is how to effectively combine information from multiple modalities. This can be done using a variety of techniques, such as fusing the features extracted from each modality, or using attention mechanisms to weight the contribution of each modality based on its importance for the task at hand.
Multimodal deep learning has many applications, including image captioning, speech recognition, natural language processing, and autonomous vehicles. By combining information from multiple modalities, multimodal deep learning can improve the accuracy and robustness of models, enabling them to perform better in real-world scenarios where multiple types of information are present.
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
Multimodal Deep Learning for Low-Resource Settings: A Vector Embedding Alignment Approach for Healthcare Applications
Towards Precision Healthcare: Robust Fusion of Time Series and Image Data
Comprehensive Multimodal Deep Learning Survival Prediction Enabled by a Transformer Architecture: A Multicenter Study in Glioblastoma
The Role of Emotions in Informational Support Question-Response Pairs in Online Health Communities: A Multimodal Deep Learning Approach
An Interpretable Adaptive Multiscale Attention Deep Neural Network for Tabular Data
HoneyBee: A Scalable Modular Framework for Creating Multimodal Oncology Datasets with Foundational Embedding Models
Research on Image Recognition Technology Based on Multimodal Deep Learning
ECC Analyzer: Extract Trading Signal from Earnings Conference Calls using Large Language Model for Stock Performance Prediction
Feature importance to explain multimodal prediction models. A clinical use case
ViTextVQA: A Large-Scale Visual Question Answering Dataset for Evaluating Vietnamese Text Comprehension in Images
In the Search for Optimal Multi-view Learning Models for Crop Classification with Global Remote Sensing Data
Describe-and-Dissect: Interpreting Neurons in Vision Networks with Language Models
Integrating Wearable Sensor Data and Self-reported Diaries for Personalized Affect Forecasting
MoPE: Mixture of Prompt Experts for Parameter-Efficient and Scalable Multimodal Fusion
A Multimodal Intermediate Fusion Network with Manifold Learning for Stress Detection
Restoring Ancient Ideograph: A Multimodal Multitask Neural Network Approach
Multimodal deep learning approach to predicting neurological recovery from coma after cardiac arrest
DeepSeek-VL: Towards Real-World Vision-Language Understanding
Cultural-Aware AI Model for Emotion Recognition
Multimodal Learning To Improve Cardiac Late Mechanical Activation Detection From Cine MR Images
Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models
Multimodal-Enhanced Objectness Learner for Corner Case Detection in Autonomous Driving
InstructIR: High-Quality Image Restoration Following Human Instructions
Multimodal Deep Learning of Word-of-Mouth Text and Demographics to Predict Customer Rating: Handling Consumer Heterogeneity in Marketing
Uncertainty-Aware Hardware Trojan Detection Using Multimodal Deep Learning
Multimodal Urban Areas of Interest Generation via Remote Sensing Imagery and Geographical Prior
Predicting the Skies: A Novel Model for Flight-Level Passenger Traffic Forecasting
Multimodal self-supervised learning for lesion localization
Integrating Chemical Language and Molecular Graph in Multimodal Fused Deep Learning for Drug Property Prediction
A graph-based multimodal framework to predict gentrification
SynthScribe: Deep Multimodal Tools for Synthesizer Sound Retrieval and Exploration
TextAug: Test time Text Augmentation for Multimodal Person Re-identification
Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge
Multimodal deep learning for mapping forest dominant height by fusing GEDI with earth observation data
HEALNet: Multimodal Fusion for Heterogeneous Biomedical Data
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.