multimodal generation
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 0
- 평가표
- 1
- 보존된 평가 기록
- 1
Multimodal generation refers to the process of generating outputs that incorporate multiple modalities, such as images, text, and sound. This can be done using deep learning models that are trained on data that includes multiple modalities, allowing the models to generate output that is informed by more than one type of data.
For example, a multimodal generation model could be trained to generate captions for images that incorporate both text and visual information. The model could learn to identify objects in the image and generate descriptions of them in natural language, while also taking into account contextual information and the relationships between the objects in the image.
Multimodal generation can also be used in other applications, such as generating realistic images from textual descriptions or generating audio descriptions of video content. By combining multiple modalities in this way, multimodal generation models can produce more accurate and comprehensive output, making them useful for a wide range of applications.
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
Multimodal Pretraining and Generation for Recommendation: A Tutorial
PCQA: A Strong Baseline for AIGC Quality Assessment Based on Prompt Condition
Food Development through Co-creation with AI: bread with a "taste of love"
PMG : Personalized Multimodal Generation with Large Language Models
3D-VLA: A 3D Vision-Language-Action Generative World Model
Retrieval-Augmented Generation for AI-Generated Content: A Survey
CLIP Model for Images to Textual Prompts Based on Top-k Neighbors
CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation
C3Net: Compound Conditioned ControlNet for Multimodal Content Generation
Towards Vision Enhancing LLMs: Empowering Multimodal Knowledge Storage and Sharing in LLMs
EasyGen: Easing Multimodal Generation with BiDiffuser and LLMs
MiniGPT-5: Interleaved Vision-and-Language Generation via Generative Vokens
Making LLaMA SEE and Draw with SEED Tokenizer
LiveChat: Video Comment Generation from Audio-Visual Multimodal Contexts
Finite Scalar Quantization: VQ-VAE Made Simple
DreamLLM: Synergistic Multimodal Comprehension and Creation
Learning to Generate Semantic Layouts for Higher Text-Image Correspondence in Text-to-Image Synthesis
Consistent Multimodal Generation via A Unified GAN Framework
SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs
Multi-modal Latent Diffusion
On Evaluating Adversarial Robustness of Large Vision-Language Models
PathAsst: A Generative Foundation AI Assistant Towards Artificial General Intelligence of Pathology
DiffBlender: Scalable and Composable Multimodal Text-to-Image Diffusion Models
DiffuSIA: A Spiral Interaction Architecture for Encoder-Decoder Text Diffusion
Accountable Textual-Visual Chat Learns to Reject Human Instructions in Image Re-creation
Grounding Language Models to Images for Multimodal Inputs and Outputs
Text2Poster: Laying out Stylized Texts on Retrieved Images
Unite and Conquer: Plug & Play Multi-Modal Synthesis using Diffusion Models
Unified Discrete Diffusion for Simultaneous Vision-Language Generation
Multimedia Generative Script Learning for Task Planning
Multimodal Generation of Novel Action Appearances for Synthetic-to-Real Recognition of Activities of Daily Living
Unconditional Image-Text Pair Generation with Multimodal Cross Quantizer
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation
GANs N' Roses: Stable, Controllable, Diverse Image to Image Translation (works for videos too!)
I Want This Product but Different : Multimodal Retrieval with Synthetic Query Expansion
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.