Scientists at the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT installedthat ambient light sensors on modern smart devices may pose a privacy risk to their owners.
These sensors collect data on the illumination of the surrounding space and help gadgets adjust the brightness of the screen. Unlike cameras, access to sensors is open to all applications, and attackers can abuse this privilege: Scientists have developed a computational imaging algorithm to obtain an image of surrounding objects from the perspective of the display. This algorithm can be used to record user commands on a touch screen, and spy applications can be browsers and video players. The ambient light sensor collects information by tracking the dynamics of light intensity, and the combination of this information with screen image data allows, using a machine learning algorithm, to identify the pixel image of the user’s hand movements.
The study authors confirmed their hypothesis in three demonstrations using an Android tablet. A human mannequin was placed in front of the device and began to touch the screen, simulating touch commands with both a human hand and its cardboard imitation – the sensor helped to recognize them. In the second demonstration, the researchers recorded a wide range of complex touch gestures, although the data processing speed was extremely low – one frame in 3.3 minutes. The third experiment showed that a video recording, be it a film or a set of short videos, can help in collecting initial data: a Tom and Jerry cartoon was launched on the screen, a human hand hovered over the sensor, and behind there was a white board that reflected light onto the device – this combination helped to restore the sequence of touch gestures.
Scientists point out that in today’s conditions the effectiveness of the attack is extremely low: images are captured at a speed of 3.3 minutes per frame, but this threat should not be underestimated. To protect themselves, they propose to take several measures: programmatically limit application access to the light sensor, and also reduce the accuracy and speed of the sensor. In addition, sensors can be placed not on the front, but on the side of the device, where it will not be able to record interaction with the touch screen.