Rewarding Episodic Visitation Discrepancy for Exploration in Reinforcement Learning
원문 초록
Exploration is critical for deep reinforcement learning in complex environments with high-dimensional observations and sparse rewards. To address this problem, recent approaches proposed to leverage intrinsic rewards to improve exploration, such as novelty-based exploration and prediction-based exploration. However, many intrinsic reward modules require sophisticated structures and representation learning, resulting in prohibitive computational complexity and unstable performance. In this paper, we propose Rewarding Episodic Visitation Discrepancy (REVD), a computation-efficient and quantified exploration method. More specifically, REVD provides intrinsic rewards by evaluating the R\'enyi divergence-based visitation discrepancy between episodes. To make efficient divergence estimation, a k-nearest neighbor estimator is utilized with a randomly-initialized state encoder. Finally, the REVD is tested on Atari games and PyBullet Robotics Environments. Extensive experiments demonstrate that REVD can significantly improves the sample efficiency of reinforcement learning algorithms and outperforms the benchmarking methods.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Generalized Adaptive Transfer Network: Enhancing Transfer Learning in Reinforcement Learning Across Domains
Deep Reinforcement Learning with Gradient Eligibility Traces
Adaptive Action Duration with Contextual Bandits for Deep Reinforcement Learning in Dynamic Environments
Margin Trader: A Reinforcement Learning Framework for Portfolio Management with Margin and Constraints
PGDQN: Preference-Guided Deep Q-Network
Multi-modal Siamese Network for Entity Alignment
토론
이 논문과 연결된 포럼 글이 아직 없습니다.