Remove Kinect depth shadow

Question

I've recently started hacking on my Kinect and I want to remove the depth shadow. The shadow is caused by the IR emitter being positioned slightly to the side of the camera, so any close object will get a big shadow and distant object less or no shadow.

The shadow length is related to the distance between the closest and the farthest spot on each side of the shadow.

My goal is to be able to map the color image correctly onto the depth. This doesn't work without processing the shadow as this picture shows:

Answer 1

Kinect Method - MapDepthFrametoColorFrame

Get the [x,y] positions in the depth frame, and use that method to fill in

Answer 2

Does the depth shadow always come out black?

If so you could use a simple method like a temporal median to calculate the background of the image (more info here: http://www.roborealm.com/help/Temporal_Median.php) and then whenever a pixel is black, set it to the background value at that pixel location.

Answer 3

I did some preliminary work on this problem a few weeks ago. My code works directly on a WriteableBitmap rather than the depth data, but if you're only doing image processing, it should work. The algorithm isn't perfect and would benefit with some more tweaking. If you update the code at all, let me know; I'd be very interested to see what you're doing!

The source code is posted on my blog:

http://richardpianka.com/2011/02/trackingni-depth-correction/

Answer 4

I don't know how it is with c# but openni c++ has a function called xnSetViewPoint() the only problem is, you lose the top 20 or so rows of imagedata due to the transformation.

the reason is due to using two different sensors, which are placed close by each other but not exactly at the same position.

Answer 5

I'm sorry to say but that shadow is caused by your body blocking the inferred dots from hitting that spot of the room so it creates a black spot... Nothing you can do but change the base background to a different color other than black so it won't be a noticeable shadow

Answer 6

The color camera and kinect depth camera dont have the same dimensions, and origin of the infra red dots are not from the same cam, its a IR projector a few cm aside from it (as that displacement is used to calculate depth).

However the solution seams easy here, your shadow data is on the left side. so you need to extend the last known color data before it went black.

And to fit it better move translate the color cam data to the right.