MagicHand: In-hand perception of object characteristics for dexterous manipulation

No Thumbnail Available
Issue Date
Li, Hui
Yihun, Yimesker S.
He, Hongsheng

Li H., Yihun Y., He H. (2018) MagicHand: In-Hand Perception of Object Characteristics for Dexterous Manipulation. In: Ge S. et al. (eds) Social Robotics. ICSR 2018. Lecture Notes in Computer Science, vol 11357. Springer, Cham


An important challenge in dexterous grasping and manipulation is to perceive the characteristics of an object such as fragility, rigidity, texture, mass and density etc. In this paper, a novel way is proposed to find these important characteristics that help in deciding grasping strategies. We collected Near-infrared (NIR) spectra of objects, classified the spectra to perceive their materials and then looked up the characteristics of the perceived material in a material-to-characteristics table. NIR spectra of six materials including ceramic, stainless steel, wood, cardboard, plastic and glass were collected using SCiO sensor. A Multi-Layer Perceptron (MLP) Neural Networks was implemented to classify the spectra. Also a material-to-characteristics table was established to map the perceived material to their characteristics. The experiment results achieve 99.96% accuracy on material recognition. In addition, a grasping experiment was performed, a robotic hand was trying to grasp two objects which shared similar shapes but made of different materials. The results showed that the robotic hand was able to improve grasping strategies based on characteristics perceived by our algorithm.

Table of Content
Click on the DOI link to access the article (may not be free).