Recently we’ve come across such an interesting Kinect research made by Japanese dev team. They made an effort by creating and producing interactive posters. The Japan team has suddenly faced the solution by inventing a Kinect tongue-control interface. Tokyo University students of Electro-Communications are working towards much more useful and important target which main goal is to help people with oral motor function disorders. Now this is a just a simple game demonstration where the user aims bullets with his tongue but in near-real time the system will show its powerful capabilities to train people with serious functional disorders to speak or swallow.
First of all the face and eyes are detected using Kinect. When both eyes are recognized, the system can estimate the nose position and based on the nose position, movement values are obtained. The minimum movement value indicates the nose tip.
So finally the tongue position is obtained. One way of training is that the tongue should be moved left and right. It is a game based on the motion. When the tongue sticks out, a bullet is shot from the player in the middle. The bullet’s trajectory depends on which way the tongue is pointing. That’s how the player aims at the targets in the game. The first problem is that detection isn’t very precise. The system isn’t very robust. So the team is considering how to improve the precision. So the training includes mouth motion combination.


Kinect For Oral Function Disorders

Those who are familiar with Kienct games will probably be wondering how it could be used to find out something as precise as tongue flicking. All in all it tends to work in its best sides with exaggerated full body motion like dance moves, football and doesn’t usually take other details such as individual finger movements. Team got it to work by face recognition, detecting the eyes, movements, in the nose and mouth, in one word, face mimic and nuance recognition. The researchers’ preliminary grade isn’t that much promisful though the system’s final goal is worthy one as more precise systems are being developed; they demand physical hardware to be affixed to the month.