With the help of Google AI and facial tracking, Android users will soon have access to a completely hands-free navigating experience that promises to revolutionize device control.
A mobile rendition of Google’s desktop product, Project Gameface, is one of the many changes and announcements the company is making during its developer keynote event today. The new virtual cursor expands the customization and manipulation capabilities of hands-free technology for both users and developers by utilizing Android accessibility services and a database of face expressions from MediaPipe’s Face Landmarks Detection API.
“Through the device’s camera, it seamlessly tracks facial expressions and head movements, translating them into intuitive and personalized control. Developers can now build applications where their users can configure their experience by customizing facial expressions, gesture sizes, cursor speed, and more,” Google said.
Google and the global accessibility solutions company Incluzza teamed up to evaluate the Gameface extension in non-gaming scenarios such as social and professional responsibilities.
In 2023, Project Gameface made its debut as an open-source, hands-free gaming mouse that let users move their heads and faces to control computer cursors. The quadriplegic viral gaming broadcaster Lance Carr worked with the technology’s designers to create a more approachable substitute for pricey head-tracking devices. Additionally, it added the gesture size option, giving users with varying levels of movement even more customization options.
“We’ve been delighted to see companies like playAbility utilize Project Gameface building blocks, like MediaPipe Blendshapes, in their inclusive software,” Google stated. “Now, we’re open sourcing more code for Project Gameface to help developers build Android applications to make every Android device more accessible.”
In order to assist those who are blind or have low vision, Google has also unveiled new AI features for TalkBack, its screenreader technology. These features will give more thorough descriptions and complete information for online images that are not labeled.
