Learning robotic grasping strategy based on natural-language object descriptions

Abstract

Given the description of an object's physical attributes, humans can determine a proper strategy and grasp an object. Accomplishing a similar feat with robotic hands is considerably challenging despite significant progress in robotics technology. Most objects of daily use are designed for human manipulation. It follows then that, humanoid robotic hands need to emulate human grasps and use a learning-based approach to learn to grasp and manipulate such objects. This paper proposes an approach to determine grasping strategy for an anthropomorphic robotic hand simply based on natural-language descriptions of an object. An artificial neural network(ANN) based learning-based approach is proposed to help a robotic hand learn suitable grasp poses starting from the natural language description of the object. Object features such as shape, size, rigidity and mass are parsed from natural-language descriptions of everyday objects using a customized natural-language processing(NLP) technique. Based on the parsed features, the most likely human-like grasp type for the given object is learned from the human grasping taxonomy using a neural network classifier. The grasping strategy generated by the proposed artificial intelligence model is evaluated both by simulation study and execution of the grasps on an AR10 robotic hand.