Common reality: An interface of human-robot communication and mutual understanding

Abstract

An interface that can share effective and comprehensive mutual understanding is critical for human-robot interaction. This paper designs a novel human-robot interaction interface that enables humans and robots to interact by their shared mutual understanding of the context. The interface superimposes robot-centered reality and human-centered reality on the working space to construct a mutual understanding environment. The common-reality interface enables humans to communicate with robots through speech and immersive touching. The mutual understanding is constructed by the user’s commands, localization of objects, recognition of objects, object semantics, and augmented trajectories. The user’s vocal commands are interpreted to formal logic, and finger touching is detected and represented by coordinates. Real-world experiments have been done to show the effectiveness of the proposed interface.