How to track eyes using Kinect SDK?

StackOverflow https://stackoverflow.com/questions/17155573

Question

The requirement is to define a rectangle around each eye in 3D space. There should be a way to track eyes using the Microsoft Kinect SDK. According to this

The Face Tracking SDK uses the Kinect coordinate system to output its 3D tracking results. The origin is located at the camera’s optical center (sensor), Z axis is pointing towards a user, Y axis is pointing up. The measurement units are meters for translation and degrees for rotation angles.

Adding

...
Debug3DShape("OuterCornerOfRightEye", faceTrackFrame.Get3DShape()
    [FeaturePoint.OuterCornerOfRightEye]);
Debug3DShape("InnerCornerRightEye", faceTrackFrame.Get3DShape()
    [FeaturePoint.InnerCornerRightEye]);
Debug3DShape("InnerCornerLeftEye", faceTrackFrame.Get3DShape()
    [FeaturePoint.InnerCornerLeftEye]);
Debug3DShape("OuterCornerOfLeftEye", faceTrackFrame.Get3DShape()
    [FeaturePoint.OuterCornerOfLeftEye]);
...
private void Debug3DShape(string s, Vector3DF v)
{
    Debug.WriteLine(s + " X " + v.X + " Y " + v.Y + " Z " + v.Z);
}

to CreateResult() in Microsoft.Kinect.Toolkit.FaceTracking prints

OuterCornerOfRightEye X -0.05728126 Y 0.04850625 Z -0.1144406
InnerCornerRightEye X -0.01584376 Y 0.04850625 Z -0.1279687
InnerCornerLeftEye X 0.01584374 Y 0.04850625 Z -0.1279687
OuterCornerOfLeftEye X 0.05728124 Y 0.04850625 Z -0.1144406

when the SDK starts tracking a face. I should be able to use these coordinates to paint a box around each eye but the Z coordinate should probably be closer to 1.0, not -0.1.. or -0.2.. (based on my setup) so I don't trust the numbers. Is the XYZ supposed to be the location of the FeaturePoint in 3D space relative to the sensor? Do I misunderstand the Kinect coordinate system? Am I using the Kinect SDK incorrectly? Does it matter that I am using an Xbox 360 Kinect Sensor? (Microsoft does not guarantee full compatibility for Kinect for Windows applications and the Xbox 360 Kinect Sensor)

Edit: Adding this

if (trackSucceeded) {
    ...
    if (headPointsObj != null) 
        for (int i = 0; i < 2; i++) 
            DebugHeadPoint(i, headPointsObj.Points);
}
private void DebugHeadPoint(int i, Vector3DF[] points) {
    if (points == null) throw new ArgumentNullException("points");
    Debug.WriteLine("HeadPoint[" + i + "] X " + points[i].X + 
        " Y " + points[i].Y + 
        " Z " + points[i].Z);
}

to FaceTracker.cs::Track() prints this

HeadPoint[0] X 0.01227657 Y -0.2290326 Z 1.319978
HeadPoint[1] X 0.00613283 Y -0.02143053 Z 1.280334
HeadPoint[0] X 0.003939687 Y -0.2297621 Z 1.319813
HeadPoint[1] X -0.003732742 Y -0.02388078 Z 1.277723
...

These numbers seem correct based on the setup. The FeaturePoints print only once but the HeadPoints print continuously while trackSucceeded. Are FeaturePoint values relative to the HeadPoint?

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top