Black region in Kinect depth image

Question

http://www.codeproject.com/Articles/151605/Kinect-and-WPF-Getting-the-raw-and-depth-image-usi

Black color in depth image comes because of large depths. But why do we have black region in the places I marked?

Answer 1

The holes on your TV/monitor are because the surface is too reflective; the infrared light from the Kinect's speckle pattern emitter doesn't get back to the Kinect's IR camera. The black halos around foreground objects like your head or shoulders are because to be scanned, a surface has to be visible to the pattern emitter and the IR camera at the same time. Surfaces immediately to the right and behind a foreground object are occluded to the pattern emitter, and surfaces immediately to the left and behind are occluded to the IR camera.

These halos get smaller when using a time-of-flight camera like the second-generation Kinect, because there the light emitter and camera are almost in the same spot.

Answer 2

I'm not a kinect expert but similar thing happens when you use too stiff height maps or projection. Kinect works by projecting points (or another texture) so if the surface normal at the point is close enough to be perpendicular to the projection vector you will lose projection because it will be stretched "to the infinity". Another problem can arise if surface has too many "noise" and projection heavily distorted. Also surfaces with transparency or too glossy.

Probably you can fill those gaps by blending neighbours regions.