The Object at Hand: Automated Editing for Mixed Reality Video Guidance from Hand-Object Interactions
원문 초록
In this paper, we concern with the problem of how to automatically extract the steps that compose real-life hand activities. This is a key competence towards processing, monitoring and providing video guidance in Mixed Reality systems. We use egocentric vision to observe hand-object interactions in real-world tasks and automatically decompose a video into its constituent steps. Our approach combines hand-object interaction (HOI) detection, object similarity measurement and a finite state machine (FSM) representation to automatically edit videos into steps. We use a combination of Convolutional Neural Networks (CNNs) and the FSM to discover, edit cuts and merge segments while observing real hand activities. We evaluate quantitatively and qualitatively our algorithm on two datasets: the GTEA\cite{li2015delving}, and a new dataset we introduce for Chinese Tea making. Results show our method is able to segment hand-object interaction videos into key step segments with high levels of precision.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Detection and tracking of fingertips for geometric transformation of objects in virtual environment
RoHOI: Robustness Benchmark for Human-Object Interaction Detection
Car Object Counting and Position Estimation via Extension of the CLIP-EBC Framework
NOCTIS: Novel Object Cyclic Threshold based Instance Segmentation
Refine Any Object in Any Scene
DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World
토론
이 논문과 연결된 포럼 글이 아직 없습니다.