Yesterday, I was invited to a private Project Natal sneak peek. The event was held at the beautiful EZ Studios in New York City. I met with a member of the Project Natal product team. She gave us a brief explanation of the Natal vision and gave us only a few technical details:
- RBG camera
- depth camera
- Current games will not be compatible with Natal
- “Project Natal” is a code name and will not be the final name of the product
- [Work in progress] Use voice commands to control services i.e. music, videos, movies
- [Work in progress] Facial recognition
- [Work in progress] Use gesturing to navigate the XBOX menu structure
- [Work in progress] Distinguishing between the primary player and a spectator
- [Work in progress] Suppressing background noise
Once we were briefed — we were given the opportunity to play a dodge ball (like) game. It was a great experience. There I was (without a controller) standing, kicking, swatting, jumping and working up a sweat. The responsiveness of the Natal system was incredible and very accurate. If I physically moved forward, back, swung faster or slower my dodge ball avatar responded in kind.
“Our advances in computer vision and audio-signal processing,” Malvar notes, “have enabled the development of Project Natal, a new level of experience for Xbox, in which user gestures and voice take the place of the standard game controller.– Rico Malvar, managing director of Microsoft Research Redmond
The key aspect of Project Natal is its ability to recognize gestures, but the system also includes the ability to respond to voice commands and facial or object recognition. These last two technologies are well established at this point. Despite some consistency issues and a literal learning curve, voice recognition is commonly found in cell phones and speech-to-text typing applications. Facial recognition, often viewed with trepidation for its big-brother security implications, is now widely available in consumer point-and-shoot cameras.