BLaFA: Bridging Language to 3D Functional Affordances for Dexterous Robotic Grasping in Tool Use

원문 초록

In industrial assembly, equipment maintenance, and everyday tool-use scenarios, robots need not only to grasp objects stably but also to establish task-appropriate contact configurations according to task intentions and tool functionality. However, existing methods lack explicit mappings from open-vocabulary instructions to 3D functional contact structures and dexterous grasp configurations, limiting robots' adaptability to diverse tools and functional tasks. To address this limitation, we propose BLaFA (Bridging Language to 3D Functional Affordances for Dexterous Robotic Grasping), a structured framework for language-driven functional dexterous grasping. First, a Knowledge-guided Language Parsing (KLP) module leverages structured knowledge to transform natural-language instructions into semantic sextuples of manipulation constraints. Second, a 3D Gaussian Splatting based hierarchical semantic representation and a Triangular Functional Point Localization (TriLocation) module are developed to localize 3D functional contact keypoints under task-tool topology constraints. Finally, a 3D Keypoint Grasp Matrix Transformation (KGT3D+) module maps the localized keypoints and semantic constraints to executable wrist poses and finger-level joint and force commands. Evaluations on 100 open-vocabulary language instructions, 14 task-tool combinations covering four functional intents, and real-world robotic experiments in representative scenes demonstrate that BLaFA can effectively translate natural-language instructions into task-aligned 3D functional contact structures and corresponding dexterous grasps, thereby enabling language-driven robotic tool use across diverse tasks. The source code will be made publicly available. Code will be publicly available at https://github.com/PopeyePxx/BLaFA.