DIAR: Diffusion-model-guided Implicit Q-learning with Adaptive Revaluation
원문 초록
We propose a novel offline reinforcement learning (offline RL) approach, introducing the Diffusion-model-guided Implicit Q-learning with Adaptive Revaluation (DIAR) framework. We address two key challenges in offline RL: out-of-distribution samples and long-horizon problems. We leverage diffusion models to learn state-action sequence distributions and incorporate value functions for more balanced and adaptive decision-making. DIAR introduces an Adaptive Revaluation mechanism that dynamically adjusts decision lengths by comparing current and future state values, enabling flexible long-term decision-making. Furthermore, we address Q-value overestimation by combining Q-network learning with a value function guided by a diffusion model. The diffusion model generates diverse latent trajectories, enhancing policy robustness and generalization. As demonstrated in tasks like Maze2D, AntMaze, and Kitchen, DIAR consistently outperforms state-of-the-art algorithms in long-horizon, sparse-reward environments.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Flow-Based Single-Step Completion for Efficient and Expressive Policy Learning
Toward Explainable Offline RL: Analyzing Representations in Intrinsically Motivated Decision Transformers
How to Provably Improve Return Conditioned Supervised Learning?
Accelerating Diffusion Models in Offline RL via Reward-Aware Consistency Trajectory Distillation
Improving Performance of Spike-based Deep Q-Learning using Ternary Neurons
Diffusion Guidance Is a Controllable Policy Improvement Operator
토론
이 논문과 연결된 포럼 글이 아직 없습니다.