Preventing shoulder surfing using randomized augmented reality keyboards

Abstract

Shoulder surfing or adversarial eavesdropping to infer users' keystrokes on physical QWERTY keyboards continues to be a serious privacy threat. Despite this, practical and efficient countermeasures against such attacks are still lacking. In this paper, we propose keyboard randomization as a simple, yet effective, countermeasure against various types of keystroke inference attacks. Our proposal consists of several keyboard randomization strategies which randomizes or changes the position of keys on the keyboard. The randomized keyboard is then projected to the typing user by means of an augmented reality wearable device. As the randomized keyboard is visually superimposed over the actual physical keyboard, and is visible only to the typing user through the augmented reality device, it acts as an effective countermeasure against both side-channel and visual-channel based keystroke inference attacks. We implement our proposed solution on a commercially available augmented reality device and conduct preliminary evaluations to validate its performance and effectiveness.