Raising Awareness for Inertial Sensors-based Keylogging on Smartphones

Nowadays, inertial sensors are embedded in almost every smartphone and are a key enabler for a wide variety of applications that build on motion. However, in the context of major mobile operating systems, these sensors do not require any permission to be used. This may cause privacy and security breaches, as motion sensors can infer a multitude of derived conditions. Our paper aims to bring attention to keylogging through inertial sensors, in which they are used to understand what the user is typing building on how the device moves or tilts. We propose a pipeline for detecting whole words by applying a combination of supervised and unsupervised methods, to identify portions of the keyboards that display similar sensor values. We then combine this method further with word frequencies in a corpus to improve the detection accuracy. We performed a data gathering campaign by distributing a mobile app to multiple users and built up a real world dataset which we used to evaluate our proposal.