Learning robotic grasping strategy based on natural-language object descriptions

Abstract

Given the description of an object's physical attributes, humans can determine a proper strategy and grasp an object. This paper proposes an approach to determine grasping strategy for an anthropomorphic robotic hand simply based on natural-language descriptions of an object. A learning-based approach is proposed to help a robotic hand learn suitable grasp poses starting from the natural language description of the object. Object features are parsed from natural-language descriptions by using a customized natural-language processing technique. The most likely grasp type for the given object is learned from the human grasping taxonomy based on the parsed features. The grasping strategy generated by the proposed approach is evaluated both by simulation study and execution of the grasps on an AR10 robotic hand.