multimodal generation
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 0
- 평가표
- 1
- 보존된 평가 기록
- 1
Multimodal generation refers to the process of generating outputs that incorporate multiple modalities, such as images, text, and sound. This can be done using deep learning models that are trained on data that includes multiple modalities, allowing the models to generate output that is informed by more than one type of data.
For example, a multimodal generation model could be trained to generate captions for images that incorporate both text and visual information. The model could learn to identify objects in the image and generate descriptions of them in natural language, while also taking into account contextual information and the relationships between the objects in the image.
Multimodal generation can also be used in other applications, such as generating realistic images from textual descriptions or generating audio descriptions of video content. By combining multiple modalities in this way, multimodal generation models can produce more accurate and comprehensive output, making them useful for a wide range of applications.
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
Benchmarking Multimodal Models for Ukrainian Language Understanding Across Academic and Cultural Domains
A Survey on Vision Autoregressive Model
Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices
Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation
ACDC: Autoregressive Coherent Multimodal Generation using Diffusion Correction
Characterizing and Efficiently Accelerating Multimodal Generation Model Inference
MM2Latent: Text-to-facial image generation and editing in GANs with multimodal assistance
PixelBytes: Catching Unified Representation for Multimodal Generation
PixelBytes: Catching Unified Embedding for Multimodal Generation
Multimodal ELBO with Diffusion Decoders
UniFashion: A Unified Vision-Language Model for Multimodal Fashion Retrieval and Generation
Learning Multimodal Latent Space with EBM Prior and MCMC Inference
Pre-trained Encoder Inference: Revealing Upstream Encoders In Downstream Machine Learning Services
MotionCraft: Crafting Whole-Body Motion with Plug-and-Play Multimodal Controls
Diffusion Models For Multi-Modal Generative Modeling
Harmonizing Visual Text Comprehension and Generation
ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generation
Empathic Grounding: Explorations using Multimodal Interaction and Large Language Models with Conversational Agents
4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities
LLMs Meet Multimodal Generation and Editing: A Survey
The Evolution of Multimodal Model Architectures
C3LLM: Conditional Multimodal Content Generation Using Large Language Models
Multimodal Pretraining and Generation for Recommendation: A Tutorial
PCQA: A Strong Baseline for AIGC Quality Assessment Based on Prompt Condition
Food Development through Co-creation with AI: bread with a "taste of love"
PMG : Personalized Multimodal Generation with Large Language Models
3D-VLA: A 3D Vision-Language-Action Generative World Model
Retrieval-Augmented Generation for AI-Generated Content: A Survey
CLIP Model for Images to Textual Prompts Based on Top-k Neighbors
CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation
C3Net: Compound Conditioned ControlNet for Multimodal Content Generation
Towards Vision Enhancing LLMs: Empowering Multimodal Knowledge Storage and Sharing in LLMs
EasyGen: Easing Multimodal Generation with BiDiffuser and LLMs
MiniGPT-5: Interleaved Vision-and-Language Generation via Generative Vokens
Making LLaMA SEE and Draw with SEED Tokenizer
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.