arXiv:2402.02017
Adaptive $Q$-Aid for Conditional Supervised Learning in Offline Reinforcement Learning
원문 초록
Offline reinforcement learning (RL) has progressed with return-conditioned supervised learning (RCSL), but its lack of stitching ability remains a limitation. We introduce $Q$-Aided Conditional Supervised Learning (QCS), which effectively combines the stability of RCSL with the stitching capability of $Q$-functions. By analyzing $Q$-function over-generalization, which impairs stable stitching, QCS adaptively integrates $Q$-aid into RCSL's loss function based on trajectory return. Empirical results show that QCS significantly outperforms RCSL and value-based methods, consistently achieving or exceeding the maximum trajectory returns across diverse offline RL benchmarks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity
DR-SAC: Distributionally Robust Soft Actor-Critic for Reinforcement Learning under Uncertainty
Offline RL with Smooth OOD Generalization in Convex Hull and its Neighborhood
From Novelty to Imitation: Self-Distilled Rewards for Offline Reinforcement Learning
Robust Bandwidth Estimation for Real-Time Communication with Offline Reinforcement Learning
Flow-Based Single-Step Completion for Efficient and Expressive Policy Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.