Google’s development of Project Gameface allows you to control the game character’s facial expressions – blink, frown and smile

A lot of work has been done lately to create controllers for people with disabilities, and Google is working on its own solution to eliminate controllers from gameplay entirely. According to media reports, as part of the Project Gameface initiative, it should be possible to control characters and processes using head movements and facial expressions.

Image source: Google

The main mechanism, as the name suggests, is facial expression control. While controllers such as the Microsoft Xbox Adaptive Controller make it much easier to get started with the game thanks to their special design and large buttons and other controls, Google’s solution provides a completely new experience.

The “mouse” that follows head movements was invented a long time ago, but such technical solutions basically register clear head movements or track blinking. Popular streamer Lance Carr, who suffers from muscular dystrophy, used such devices before a fire destroyed his gaming hardware. At the same time, the incident also had positive effects – after that, Google engineers seriously began to improve the technology.

The result was Project Gameface. “Gaming Mouse” is open source software that requires no hand controls and uses a regular webcam pointed at the user’s face to issue commands. Machine learning allows you to track 468 unique points on the face, which allows GameFace to accurately recognize not only head movement but also facial expressions. For example, this or that facial expression is converted into a mouse click or other commands. Project Gameface even allows you to customize controls for individual facial expressions, so even those who can barely control facial muscles can benefit from the technology.

Image source: Google

According to Google, the GameFace project needs to be completed and is not yet ready for mass deployment. However, the sources are already available available on GitHub those who wish to contribute to ongoing work. It is possible that the possibility of such a control – in addition to classic controls – can also be useful for healthy people.