SparseDM: Toward Sparse Efficient Diffusion Models
원문 초록
Diffusion models represent a powerful family of generative models widely used for image and video generation. However, the time-consuming deployment, long inference time, and requirements on large memory hinder their applications on resource constrained devices. In this paper, we propose a method based on the improved Straight-Through Estimator to improve the deployment efficiency of diffusion models. Specifically, we add sparse masks to the Convolution and Linear layers in a pre-trained diffusion model, then transfer learn the sparse model during the fine-tuning stage and turn on the sparse masks during inference. Experimental results on a Transformer and UNet-based diffusion models demonstrate that our method reduces MACs by 50% while maintaining FID. Sparse models are accelerated by approximately 1.2x on the GPU. Under other MACs conditions, the FID is also lower than 1 compared to other methods.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Minute-Long Videos with Dual Parallelisms
VideoMAR: Autoregressive Video Generatio with Continuous Tokens
GigaVideo-1: Advancing Video Generation via Automatic Feedback with 4 GPU-Hours Fine-Tuning
Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion
Astraea: A GPU-Oriented Token-wise Acceleration Framework for Video Diffusion Transformers
APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.