How to find the Joint coordinates(X,Y,Z) ,also how to draw a locus of the tracked joint?

Question

I am trying to develop a logic to recognize a circle which is made by users right hand, I got the code to draw the skeleton and track from the sample code,

private void SensorSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
    {
        Skeleton[] skeletons = new Skeleton[0];

        using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
        {
            if (skeletonFrame != null)
            {
                skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength];
                skeletonFrame.CopySkeletonDataTo(skeletons);
            }
        }

        using (DrawingContext dc = this.drawingGroup.Open())
        {
            // Draw a transparent background to set the render size
            dc.DrawRectangle(Brushes.Black, null, new Rect(0.0, 0.0, RenderWidth, RenderHeight));

            if (skeletons.Length != 0)
            {
                foreach (Skeleton skel in skeletons)
                {
                    RenderClippedEdges(skel, dc);

                    if (skel.TrackingState == SkeletonTrackingState.Tracked)
                    {
                        this.DrawBonesAndJoints(skel, dc);
                    }
                    else if (skel.TrackingState == SkeletonTrackingState.PositionOnly)
                    {
                        dc.DrawEllipse(
                        this.centerPointBrush,
                        null,
                        this.SkeletonPointToScreen(skel.Position),
                        BodyCenterThickness,
                        BodyCenterThickness);
                    }
                }
            }

            // prevent drawing outside of our render area
            this.drawingGroup.ClipGeometry = new RectangleGeometry(new Rect(0.0, 0.0, RenderWidth, RenderHeight));
        }
    }

What I want to do now is to track the coordinates of users right hand for gesture recognition,

Here is how I am planning to get the job done:

1. Start the gesture
2. Draw the circled gesture, Make sure to store the coordinates for start and then keep noting the coordinates for every 45 degree shift of the Joint from the start, for 8 octants we will get 8 samples.
3. For making a decision that a circle was drawn we can just check the relation ship between the eight samples.

Also, in the depthimage I want to show the locus of the drawn gesture, so as the handpoint moves it leaves a trace behind so at the end we will get a figure which was drawn by an user. I have no idea how to achieve this.

Answer 1

Coordinates for each joint are available for each tracked skeleton during each SkeletonFrameReady event. Inside your foreach loop...

foreach (Skeleton skeleton in skeletons) {
    // get the joint
    Joint rightHand = skeleton.Joints[JointType.HandRight];

    // get the individual points of the right hand
    double rightX = rightHand.Position.X;
    double rightY = rightHand.Position.Y;
    double rightZ = rightHand.Position.Z;
}

You can look at the JointType enum to pull out any of the joints and work with the individual coordinates.

To draw your gesture trail you can use the DrawContext you have in your example or use another way to draw a Path onto the visual layer. With your x/y/z values, you would need to scale them to the window coordinates. The "Coding4Fun" library offers a pre-built function to do it; alternatively you can write your own, for example:

private static double ScaleY(Joint joint)
{
    double y = ((SystemParameters.PrimaryScreenHeight / 0.4) * -joint.Position.Y) + (SystemParameters.PrimaryScreenHeight / 2);
    return y;
}

private static void ScaleXY(Joint shoulderCenter, bool rightHand, Joint joint, out int scaledX, out int scaledY)
{
    double screenWidth = SystemParameters.PrimaryScreenWidth;

    double x = 0;
    double y = ScaleY(joint);

    // if rightHand then place shouldCenter on left of screen
    // else place shouldCenter on right of screen
    if (rightHand)
    {
        x = (joint.Position.X - shoulderCenter.Position.X) * screenWidth * 2;
    }
    else
    {
        x = screenWidth - ((shoulderCenter.Position.X - joint.Position.X) * (screenWidth * 2));
    }


    if (x < 0)
    {
        x = 0;
    }
    else if (x > screenWidth - 5)
    {
        x = screenWidth - 5;
    }

    if (y < 0)
    {
        y = 0;
    }

    scaledX = (int)x;
    scaledY = (int)y;
}