SIGMA: An Open-Source Interactive System for Mixed-Reality Task Assistance Research
원문 초록
We introduce an open-source system called SIGMA (short for "Situated Interactive Guidance, Monitoring, and Assistance") as a platform for conducting research on task-assistive agents in mixed-reality scenarios. The system leverages the sensing and rendering affordances of a head-mounted mixed-reality device in conjunction with large language and vision models to guide users step by step through procedural tasks. We present the system's core capabilities, discuss its overall design and implementation, and outline directions for future research enabled by the system. SIGMA is easily extensible and provides a useful basis for future research at the intersection of mixed reality and AI. By open-sourcing an end-to-end implementation, we aim to lower the barrier to entry, accelerate research in this space, and chart a path towards community-driven end-to-end evaluation of large language, vision, and multimodal models in the context of real-world interactive applications.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Beyond the Monitor: Mixed Reality Visualization and AI for Enhanced Digital Pathology Workflow
Single View Garment Reconstruction Using Diffusion Mapping Via Pattern Coordinates
Blind Augmentation: Calibration-free Camera Distortion Model Estimation for Real-time Mixed-reality Consistency
Gravity-aligned Rotation Averaging with Circular Regression
MaRINeR: Enhancing Novel Views by Matching Rendered Images with Nearby References
Hearing Anything Anywhere
토론
이 논문과 연결된 포럼 글이 아직 없습니다.