Implicit Diffusion: Efficient Optimization through Stochastic Sampling
원문 초록
We present a new algorithm to optimize distributions defined implicitly by parameterized stochastic diffusions. Doing so allows us to modify the outcome distribution of sampling processes by optimizing over their parameters. We introduce a general framework for first-order optimization of these processes, that performs jointly, in a single loop, optimization and sampling steps. This approach is inspired by recent advances in bilevel optimization and automatic implicit differentiation, leveraging the point of view of sampling as optimization over the space of probability distributions. We provide theoretical guarantees on the performance of our method, as well as experimental results demonstrating its effectiveness in real-world settings.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
GuiLoMo: Allocating Expert Number and Rank for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectors
MISLEADER: Defending against Model Extraction with Ensembles of Distilled Models
Learning Where to Learn: Training Distribution Selection for Provable OOD Performance
GCAL: Adapting Graph Models to Evolving Domain Shifts
HBO: Hierarchical Balancing Optimization for Fine-Tuning Large Language Models
Optimizing Decomposition for Optimal Claim Verification
토론
이 논문과 연결된 포럼 글이 아직 없습니다.