Researchers have developed connected glasses capable of detecting silently articulated words. They could be used to control your smartphone, or even be combined with a voice synthesizer to give a voice to mute people.
You will also be interested
[EN VIDÉO] Humanity shares 70% of its facial expressions The approximately seven billion humans inhabiting the Earth would have another point in common. Finally, 16…
Soon, your eyeglasses might be able to control your smartphone. Researchers at Cornelle University in the United States have integrated a sonar system on connected glasses that can detect lip movements. Called EchoSpeech, they are extremely discreet, and even use a commercial eyeglass frame.
The system is made up of two miniature speakers under one eye that emit ultrasound towards the mouth. Two microphones, placed under the other eye, record the echoes. The suit allows four different signals to be recorded which are then analyzed by a deep learning system which infers mouth movements. According to the researchers, with only two training sessions, the EchoSpeech can already recognize 31 commands.
A system that only requires a smartphone
Choosing a sound system over cameras offers several advantages. The components are cheaper and smaller, making the glasses much more discreet and light, and therefore more comfortable to wear. Autonomy is also better. The EchoSpeech operate for about ten hours between two recharges, where a similar solution with cameras would be limited to only 30 minutes. In addition, audio data is much easier to process. The glasses send all the data via Bluetooth to a smartphone which can process it in real time. Finally, this solution is more respectful of privacy than a camera that records the face. The data is processed locally and the glasses filter out the lower frequencies, thus avoiding recording any conversation in the surroundings.
Eventually, EchoSpeech could make it possible to dictate text in places where it is not possible to speak, for example in a library or in a noisy place such as a restaurant or a concert hall. In addition, it could be combined with a voice synthesis system to give a voice to mute people and allow them to communicate without sign language. Researchers are currently working on recognizing facial expressions, as well as eye and upper body movements. Such a system could in particular be integrated into virtual reality headsets to animate one’s avatar.