Variational Adaptive Noise and Dropout towards Stable Recurrent Neural Networks
원문 초록
This paper proposes a novel stable learning theory for recurrent neural networks (RNNs), so-called variational adaptive noise and dropout (VAND). As stabilizing factors for RNNs, noise and dropout on the internal state of RNNs have been separately confirmed in previous studies. We reinterpret the optimization problem of RNNs as variational inference, showing that noise and dropout can be derived simultaneously by transforming the explicit regularization term arising in the optimization problem into implicit regularization. Their scale and ratio can also be adjusted appropriately to optimize the main objective of RNNs, respectively. In an imitation learning scenario with a mobile manipulator, only VAND is able to imitate sequential and periodic behaviors as instructed. https://youtu.be/UOho3Xr6A2w
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Interpretable Bayesian Tensor Network Kernel Machines with Automatic Rank and Feature Selection
Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference
Variational Inference with Mixtures of Isotropic Gaussians
Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism
Solving the Job Shop Scheduling Problem with Graph Neural Networks: A Customizable Reinforcement Learning Environment
Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.