MagicHand: In-hand perception of object characteristics for dexterous manipulation

Abstract

An important challenge in dexterous grasping and manipulation is to perceive the characteristics of an object such as fragility, rigidity, texture, mass and density etc. In this paper, a novel way is proposed to find these important characteristics that help in deciding grasping strategies. We collected Near-infrared (NIR) spectra of objects, classified the spectra to perceive their materials and then looked up the characteristics of the perceived material in a material-to-characteristics table. NIR spectra of six materials including ceramic, stainless steel, wood, cardboard, plastic and glass were collected using SCiO sensor. A Multi-Layer Perceptron (MLP) Neural Networks was implemented to classify the spectra. Also a material-to-characteristics table was established to map the perceived material to their characteristics. The experiment results achieve 99.96% accuracy on material recognition. In addition, a grasping experiment was performed, a robotic hand was trying to grasp two objects which shared similar shapes but made of different materials. The results showed that the robotic hand was able to improve grasping strategies based on characteristics perceived by our algorithm.