How to capture motion data in Kinect SDK V1.7 (Avateering XNA)?

StackOverflow https://stackoverflow.com/questions/21796703

Pergunta

I am new to the kinect sdk v1.7.

I want to know how to capture motion data from the sample .(http://msdn.microsoft.com/en-us/library/jj131041.aspx)

so, how can I make a procedure that can capture the skeleton data into file ? (record)

then , read the file back to the sample program and model it.(play)?

my idea is record the skeleton data into file , then get the skeleton data from the file and let the Avatar play .

I can do what I want in another sample program . (http://msdn.microsoft.com/en-us/library/hh855381) , cause the sample program only draw the lines and skeleton points.

For example , 

00001 00:00:00.0110006@353,349,354,332,358,249,353,202,310,278,286,349,269,407,266,430,401,279,425,349,445,408,453,433,332,369,301,460,276,539,269,565,372,370,379,466,387,548,389,575,

00002 00:00:00.0150008@352,349,353,332,356,249,352,202,309,278,284,349,266,406,263,430,398,279,424,349,445,408,453,433,331,369,301,461,277,541,271,566,371,371,379,466,387,548,390,575,

[frame no.][timestamp]@[skeleton position coordinates]

in this example , i assume the skeleton position is the Joint Id order.

thanks (forgive my poor english).

Foi útil?

Solução

You can use a StreamWriter, initialize it at the selected path, then for each frame, increase the frame counter, write it to the file, write the timestamp to the file, then loop through the joints and write them to the file. I would do this as follows:

using System.IO;

StreamWriter writer = new StreamWriter(@path);
int frames = 0;

...

void AllFramesReady(object sender, AllFramesReadyEventArgs e)
{
    frames++;
    using (SkeletonFrame sFrame = e.OpenSkeletonFrameData())
    {
        if (sFrame == null)
            return;

        skeletonFrame.CopySkeletonDataTo(skeletons);

        Skeleton skeleton = (from s in skeletons
                                where s.TrackingState == SkeletonTrackingState.Tracked
                                select s);
        if (skeleton == null)
            return;

        if (skeleton.TrackingState == SkeletonTrackingState.Tracked)
        {
            writer.Write("{0} {1}@", frames, timestamp);//I dont know how you want to do this
            foreach (Joint joint in skeleton.Joints)
            {
                writer.Write(joint.Position.X + "," + joint.Position.Y + "," joint.Position.Z + ",");
            }
            writer.Write(Environment.NewLine);
        }
    }
}

Then to read from the file:

StreamReader reader = new StreamReader(@path);
int frame = -1;
JointCollection joints;

...

string[] lines = reader.ReadAllLines();

...

void AllFramesReady(object sender, AllFramesReadyEventArgs e)
{
    canvas.Children.Clear();
    string[] coords = lines[frame].Split('@')[1].Split(',');
    int jointIndex = 0;
    for (int i = 0; i < coords.Length; i += 3)
    {
        joints[jointIndex].Position.X = int.Parse(coords[i]);
        joints[jointIndex].Position.Y = int.Parse(coords[i + 1]);
        joints[jointIndex].Position.X = int.Parse(coords[i + 2]);
        jointIndex++;
    }

    DepthImageFrame depthFrame = e.OpenDepthImageFrame();
    canvas.Children.Add(GetBodySegment(joints, brush, new JointType[] { JointType.HipCenter, JointType.Spine, JointType.ShoulderCenter, JointType.Head }, depthFrame, canvas));
    canvas.Children.Add(GetBodySegment(joints, brush, new JointType[] { JointType.ShoulderCenter, JointType.ShoulderLeft, JointType.ElbowLeft, JointType.WristLeft, JointType.HandLeft }, depthFrame, canvas));
    canvas.Children.Add(GetBodySegment(joints, brush, new JointType[] { JointType.ShoulderCenter, JointType.ShoulderRight, JointType.ElbowRight, JointType.WristRight, JointType.HandRight }, depthFrame, canvas));
    canvas.Children.Add(GetBodySegment(joints, brush, new JointType[] { JointType.HipCenter, JointType.HipLeft, JointType.KneeLeft, JointType.AnkleLeft, JointType.FootLeft }, depthFrame, canvas));
    canvas.Children.Add(GetBodySegment(joints, brush, new JointType[] { JointType.HipCenter, JointType.HipRight, JointType.KneeRight, JointType.AnkleRight, JointType.FootRight }, depthFrame, canvas));
    depthFrame.Dispose();

    frame++;
}

Point GetDisplayPosition(Joint joint, DepthImageFrame depthFrame, Canvas skeleton)
{
    float depthX, depthY;
    KinectSensor sensor = KinectSensor.KinectSensors[0];
    DepthImageFormat depthImageFormat = sensor.DepthStream.Format;
    DepthImagePoint depthPoint = sensor.CoordinateMapper.MapSkeletonPointToDepthPoint(joint.Position, depthImageFormat);

    depthX = depthPoint.X;
    depthY = depthPoint.Y;

    depthX = Math.Max(0, Math.Min(depthX * 320, 320));
    depthY = Math.Max(0, Math.Min(depthY * 240, 240));

    int colorX, colorY;
    ColorImagePoint colorPoint = sensor.CoordinateMapper.MapDepthPointToColorPoint(depthImageFormat, depthPoint, ColorImageFormat.RgbResolution640x480Fps30);
    colorX = colorPoint.X;
    colorY = colorPoint.Y;

    return new System.Windows.Point((int)(skeleton.Width * colorX / 640.0), (int)(skeleton.Height * colorY / 480));
}

Polyline GetBodySegment(Joint[] joints, Brush brush, JointType[] ids, DepthImageFrame depthFrame, Canvas canvas)
{
    PointCollection points = new PointCollection(ids.Length);
    for (int i = 0; i < ids.Length; ++i)
    {
        points.Add(GetDisplayPosition(joints[i], depthFrame, canvas));
    }
    Polyline polyline = new Polyline();
    polyline.Points = points;
    polyline.Stroke = brush;
    polyline.StrokeThickness = 5;
    return polyline;
}

Of course, this only works in wpf. You will just need to change from using the code:

    DepthImageFrame depthFrame = e.OpenDepthImageFrame();
    canvas.Children.Add(GetBodySegment(joints, brush, new JointType[] { JointType.HipCenter, JointType.Spine, JointType.ShoulderCenter, JointType.Head }, depthFrame, canvas));
    canvas.Children.Add(GetBodySegment(joints, brush, new JointType[] { JointType.ShoulderCenter, JointType.ShoulderLeft, JointType.ElbowLeft, JointType.WristLeft, JointType.HandLeft }, depthFrame, canvas));
    canvas.Children.Add(GetBodySegment(joints, brush, new JointType[] { JointType.ShoulderCenter, JointType.ShoulderRight, JointType.ElbowRight, JointType.WristRight, JointType.HandRight }, depthFrame, canvas));
    canvas.Children.Add(GetBodySegment(joints, brush, new JointType[] { JointType.HipCenter, JointType.HipLeft, JointType.KneeLeft, JointType.AnkleLeft, JointType.FootLeft }, depthFrame, canvas));
    canvas.Children.Add(GetBodySegment(joints, brush, new JointType[] { JointType.HipCenter, JointType.HipRight, JointType.KneeRight, JointType.AnkleRight, JointType.FootRight }, depthFrame, canvas));
    depthFrame.Dispose();

To how the avateering sample animates the model, you could even create a new Skeleton and copy joints to Skeleton.Joints, then just pass that skeleton as the "detected" skeleton. Note you would need to change any other needed variables that are required for functions used in this sample. I am unfamiliar with the sample so I can't give specific method names, but you can just replace the global Skeleton with the one you created at the beginning and update every frame. So I would recommend this:

//in the game class (AvateeringXNA.cs)
StreamReader reader = new StreamReader(@path);
int frame = -1;
JointCollection joints;
Skeleton recorded = new Skeleton();

...

string[] lines = reader.ReadAllLines();

...

void Update(...)
{
    string[] coords = lines[frame].Split('@')[1].Split(',');
    int jointIndex = 0;
    for (int i = 0; i < coords.Length; i += 3)
    {
        joints[jointIndex].Position.X = int.Parse(coords[i]);
        joints[jointIndex].Position.Y = int.Parse(coords[i + 1]);
        joints[jointIndex].Position.X = int.Parse(coords[i + 2]);
        jointIndex++;
    }

    recorded.Joints = joints;

    ...

    //preform necessary methods, except with recorded skeleton instead of detected, I think it is:
    this.animator.CopySkeleton(recorded);
    this.animator.FloorClipPlane = skeletonFrame.FloorClipPlane;

    // Reset the filters if the skeleton was not seen before now
    if (this.skeletonDetected == false)
    {
        this.animator.Reset();
    }

    this.skeletonDetected = true;
    this.animator.SkeletonVisible = true;

    ...

    frame++;
}

EDIT

When you read the initial floor clip plane (clipPlanes[0]) it will get the entire frame information up to the first space. See below to see how it will split and how I would read it:

var newFloorClipPlane = Tuple.Create(Single.Parse(clipPlanes[2]), Single.Parse(clipPlanes[3]), Single.Parse(clipPlanes[4]), Single.Parse(clipPlanes[5]));

Here is how you lay out the frames:

frame# timestam@joint1Posx,joint1posy,joint1posz,...jointNPosx,jointNposy,jointNposz floorX floorY floorZ floorW

Here is the array produced by `.Split(' ')

["frame#", "timestam@joint1Posx,joint1posy,joint1posz,...jointNPosx,jointNposy,jointNposz", "floorX", "floorY", "floorZ", "floorW"]

Therefore, with an example input of:

00000002 10112@10,10,10... 11 12 13 14

With your code you will get:

[2, 10112101010..., 11, 12]

With corrected indexes from my code:

[11, 12, 13, 14]

Put this line of into a console application really fast and see what it outputs:

Console.WriteLine(Convert.ToSingle("10,10"));

The output is 1010 For what you are trying to accomplish, this creates the wrong floor clip plane. You need the proper indexes for what you are trying to achieve.

note: I changed Convert.ToSingle to Single.Parse because it is better practice and in the stack trace they both preform the same functions

Outras dicas

Hey why don't you use the csv method to write all the joints data in terms of excel file. It would help you to analyse them at a later stage. I have customized my code for putting them in csv format which helped me to analyse at later stage of data. You can write a separate file in your project which would export all the skeleton data 

public void CoordinatesExportToCSV(Skeleton data)
    {
        if (!TimeRecorded)
        {
            startTime = DateTime.Now;
            TimeRecorded = true;
        }
        recordedSamples[1]++;
        if (!titles)
        {
            sw1.Write("Counter,Time,Clipped Edges,");
            foreach (Joint joint in data.Joints)
            {
                sw1.Write(joint.JointType.ToString()+",");
            }
            titles = true;
        }
        else
        {
            double a = DateTime.Now.TimeOfDay.TotalSeconds - startTime.TimeOfDay.TotalSeconds;
            sw1.Write(recordedSamples[1] + "," + a + "," + data.ClippedEdges);

            foreach (Joint joint in data.Joints)
            {
                sw1.Write(joint.Position.X + "|" + joint.Position.Y + "|" + joint.Position.Z+",");
            }
        }
        sw1.WriteLine();
    }
Licenciado em: CC-BY-SA com atribuição
Não afiliado a StackOverflow
scroll top