Augmented Reality Mask using Facial Recognition on Xbox Kinect with Kinect for Windows SDK

Question

I am using an XBox Kinect with the Kinect for Windows SDK. I want to make an application that will augment a 3D mask (a 3D model of a mask made in 3DS Max) onto the face of anyone using the application. The application will be used in an exhibit locally. I have not tried much because I don't know where to start. So what I want to know is, is it currently possible to augment a 3DS Max model onto a live video stream using the facial recognition and skeletal tracking features in the newest Kinect for Windows SDK, and if so, how/where should I start trying to do/implement this? Any point in the right direction would be great. Thank you! PS And yes, I have read the UI guidelines and the facial documentation. My problem is one of not knowing where to start programming, not one of not understanding the fundamental concepts. Thanks!

Answer 1

If you are serious about getting into developing for the Kinect I would recommend getting this book: http://www.amazon.com/Programming-Kinect-Windows-Software-Development/dp/0735666814

This goes through developing with the Kinect for Windows SDK from the ground up. There is a face tracking and an augmented reality example so I'm pretty sure you will be able to achieve your goal quite easily.

All the code from the book is here: http://kinecttoolbox.codeplex.com/

---

Alternatively, there is an example here which pretty much is what you want to achieve: http://www.codeproject.com/Articles/213034/Kinect-Getting-Started-Become-The-Incredible-Hulk

It is developed using the Beta version of the SDK, but the same priciples apply.

---

You can also check out the quick start videos here:

http://channel9.msdn.com/Series/KinectQuickstart

---

In summary, based on my own experience, I would spend some time going through the beginner examples either in the vides or the book (I found the book very good) just to get familiar with how to setup a simple Kinect project and how the different parts of the SDK work.

When you have developed some throwaway apps with the Kinect, I would then try tackling your project (although, the Incredible Hulk project above should get you most the way there!)

Best of luck with your project