Reading a depth png image from a dataset

Question

We are trying to use this American Sign Language dataset. This dataset has pictures of American Sign Language letters, both RGB and the Depth images.

I downloaded the dataset from the link. The RGB images seems fine, but the depth images are fully solid black. Something is wrong.

Since all the dataset is big, and it takes time to download all of them; I'm uploading an example RGB image and an example depth image here:

Since the depth images should have the depth data, I expect it to have float values (They say they used Kinect and Kinect provides float values). How can I read these float pixels using C#? I tried the following:

Bitmap bmp = new Bitmap("depth_0_0002.png");
int R = bmp.GetPixel(0,0).R;
int G = bmp.GetPixel(0,0).G;
int B = bmp.GetPixel(0,0).B;

However, I need float pixels, these are integer and they have nonsense values.

Do I need to include a 3rd party library?

Answer 1

I've tried it myself. Normally the depth datas are 16bit values. The 13 high-order bits contain the distance and the 3 low-order bits contain the user segmentation map.

The user segmentation map is only built if skeleton tracking is active, which I believe was not in your example. Although the rgb values are 24bit it seems to work. I get an image from the segmented hand.

Bitmap bmpOrg = new Bitmap("bKawM.png");
Bitmap bmp = new Bitmap(106, 119);

for (int i = 0; i < 106;i++ )
{
    for (int j = 0; j < 119;j++ )
    {
        Color rgb = bmpOrg.GetPixel(i, j);

        int bit24 = (rgb.B << 16 + rgb.G << 8 + rgb.R);
        int user = bit24 & 0x07;
        int realDepth = bit24 >> 3;

        bmp.SetPixel(i, j, Color.FromArgb(realDepth));
    }
}

pictureBox1.Image = bmp;

My output:

I've played with it again. First I increased the brightness and contrast in Photoshop. So the rgb values are usable if you don't need the real depth values in millimeters.

Then I tried to get the 16bit values from image with WPF because the image is 16bit grayscale encoded.

Stream imageStreamSource = new FileStream("bKawM.png", FileMode.Open, FileAccess.Read, FileShare.Read);
PngBitmapDecoder decoder = new PngBitmapDecoder(imageStreamSource, BitmapCreateOptions.PreservePixelFormat, BitmapCacheOption.Default);
BitmapSource bitmapSource = decoder.Frames[0];

int height = bitmapSource.PixelHeight;
int width = bitmapSource.PixelWidth;
int stride = width * ((bitmapSource.Format.BitsPerPixel + 7) / 8);

byte[] bytes = new byte[height * stride];
bitmapSource.CopyPixels(bytes, stride, 0);

for (int x = 0; x < width; x++)
{
    for (int y = 0; y < height; y++)
    {
        byte low = bytes[y * stride + x + 0];
        byte high = bytes[y * stride + x + 1];

        ushort bit16 = (ushort)((high << 8) | low);

        int user = bit16 & 0x07;
        int realDepth = bit16 >> 3;

    }
}

I created a new image with the depth values and it looked very strange. I don't find any information what data the image contains. I don't know if it contains the userdata (3 bits) or if the depth is converted somehow before saving to file.