Data-efficient visuomotor policy training using reinforcement learning and generative models
원문 초록
We present a data-efficient framework for solving visuomotor sequential decision-making problems which exploits the combination of reinforcement learning (RL) and latent variable generative models. Our framework trains deep visuomotor policies by introducing an action latent variable such that the feed-forward policy search can be divided into three parts: (i) training a sub-policy that outputs a distribution over the action latent variable given a state of the system, (ii) unsupervised training of a generative model that outputs a sequence of motor actions conditioned on the latent action variable, and (iii) supervised training of the deep visuomotor policy in an end-to-end fashion. Our approach enables safe exploration and alleviates the data-inefficiency problem as it exploits prior knowledge about valid sequences of motor actions. Moreover, we provide a set of measures for evaluation of generative models such that we are able to predict the performance of the RL policy training prior to the actual training on a physical robot. We define two novel measures of disentanglement and local linearity for assessing the quality of latent representations, and complement them with existing measures for assessment of the learned distribution. We experimentally determine the characteristics of different generative models that have the most influence on performance of the final policy training on a robotic picking task.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
A Survey of Continual Reinforcement Learning
Flow-Based Single-Step Completion for Efficient and Expressive Policy Learning
Actor-Critic Deep Reinforcement Learning for Solving Job Shop Scheduling Problems
Constructing Non-Markovian Decision Process via History Aggregator
UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making
Adaptive Action Duration with Contextual Bandits for Deep Reinforcement Learning in Dynamic Environments
토론
이 논문과 연결된 포럼 글이 아직 없습니다.