Understanding plasticity in neural networks
원문 초록
Plasticity, the ability of a neural network to quickly change its predictions in response to new information, is essential for the adaptability and robustness of deep reinforcement learning systems. Deep neural networks are known to lose plasticity over the course of training even in relatively simple learning problems, but the mechanisms driving this phenomenon are still poorly understood. This paper conducts a systematic empirical analysis into plasticity loss, with the goal of understanding the phenomenon mechanistically in order to guide the future development of targeted solutions. We find that loss of plasticity is deeply connected to changes in the curvature of the loss landscape, but that it often occurs in the absence of saturated units. Based on this insight, we identify a number of parameterization and optimization design choices which enable networks to better preserve plasticity over the course of training. We validate the utility of these findings on larger-scale RL benchmarks in the Arcade Learning Environment.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Generalized Adaptive Transfer Network: Enhancing Transfer Learning in Reinforcement Learning Across Domains
Adaptive Action Duration with Contextual Bandits for Deep Reinforcement Learning in Dynamic Environments
Unraveling the Rainbow: can value-based methods schedule?
Pay Attention to What and Where? Interpretable Feature Extractor in Vision-based Deep Reinforcement Learning
Automatic Reward Shaping from Confounded Offline Data
Adventurer: Exploration with BiGAN for Deep Reinforcement Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.