Learning Actions from Human Demonstration Video for Robotic Manipulation
원문 초록
Learning actions from human demonstration is an emerging trend for designing intelligent robotic systems, which can be referred as video to command. The performance of such approach highly relies on the quality of video captioning. However, the general video captioning methods focus more on the understanding of the full frame, lacking of consideration on the specific object of interests in robotic manipulations. We propose a novel deep model to learn actions from human demonstration video for robotic manipulation. It consists of two deep networks, grasp detection network (GNet) and video captioning network (CNet). GNet performs two functions: providing grasp solutions and extracting the local features for the object of interests in robotic manipulation. CNet outputs the captioning results by fusing the features of both full frames and local objects. Experimental results on UR5 robotic arm show that our method could produce more accurate command from video demonstration than state-of-the-art work, thereby leading to more robust grasping performance.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
UGC-VideoCaptioner: An Omni UGC Video Detail Caption Model and New Benchmarks
video-SALMONN 2: Captioning-Enhanced Audio-Visual Large Language Models
FLASH: Latent-Aware Semi-Autoregressive Speculative Decoding for Multimodal Tasks
FocusedAD: Character-centric Movie Audio Description
Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Prompting
Pretrained Image-Text Models are Secretly Video Captioners
토론
이 논문과 연결된 포럼 글이 아직 없습니다.