Unpacking RGB values in a Point Cloud Data Set

Question

This is regarding unpacking the encoded rgb values in a pcl file. I did this with the procedure described in the pcl documentation, but the unpacked rgb values I get are not quite correct. When I plot them with R the representation given does not correspond to the colors in the real setting (I am to a certain degree sure the problem is not with the way it was plotted with R).

For example in the image attached the area demarcated should have colors gray and blue (two chairs and a table).

Source pcl file could be found at: https://docs.google.com/open?id=0Bz5-HVcDiF6SanBZU0JWVmJwWHM and the file with the unpacked color values at: https://docs.google.com/open?id=0Bz5-HVcDiF6SV2pYQ0xUbTAwVmM. Also following is the code used for unpacking the color values in a c plus plus setting:

uint32_t rgbD = *reinterpret_cast<int*>(&kinectValue);

    uint16_t rD = (rgbD >> 16) & 0x0000ff;
    uint16_t gD = (rgbD >> 8)  & 0x0000ff;
    uint16_t bD = (rgbD)       & 0x0000ff;

I would really appreciate if you could let me know where I have gone wrong.

Update:

Following is the R code snippet I used in plotting the values in 3D:

library(rgl)
pcd <- read.table(file.choose(),sep="")
names(pcd) <- c("x","y","z","r","g","b")
plot3d(pcd$x,pcd$y,pcd$z,col=rgb(pcd$r,pcd$g,pcd$b,maxColorValue=255))

Update:

Following is the code I used to read data, in C++:

/*
Reads in a file from Kinect with unpacked color values, filter the color value component and
sends it to be unpacked
*/
int fileRead(){

  string line;
  int lineNum = 0;

  ifstream myfile ("res/OnePerson4.pcd");

  if (myfile.is_open())
  {
    while ( myfile.good() )
    {
      lineNum++;
      getline (myfile,line);

       // Exclude the header information in the kinect file from the unpacking process
      //if(lineNum > 10 && lineNum <20){//This for loop is activated when testing
      if(lineNum > 10){
        //Test code to extract the x,y,z values
        string xyzvalFromKinectStr = line.substr(0,line.find_last_of(' '));
        //cout<<xyzvalFromKinectStr<<"\n";
        //Extract the packed rgb value
        string valFromKinectStr = line.substr(line.find_last_of(' '));
        double kinectVal = ::atof(valFromKinectStr.c_str());
        kinectToRgb(kinectVal, xyzvalFromKinectStr);

      }
    }
    myfile.close();
  }
  else
  {
      cout << "Unable to open file";
  }

  return 0;
}

Answer 1

Here's my working solution. First I ran your input through grep to filter out NANs in coordinates:

$ grep -v nan OnePerson4.pcd > OnePerson4.pcd.filtered

Then I extracted the data via C++ code like this:

#include <stdio.h>

int main()
{
    if (FILE *f = fopen("OnePerson4.pcd.filtered", "rt"))
    {
        for (;;)
        {
            float x = 0;
            float y = 0;
            float z = 0;
            float color_float = 0;
            if (fscanf(f, "%f %f %f %f", &x, &y, &z, &color_float) != 4)
            {
                break;
            }

            unsigned color = *(unsigned const *)&color_float;
            unsigned r = color & 0xff;
            unsigned g = (color >> 8) & 0xff;
            unsigned b = (color >> 16) & 0xff;
            printf("%f,%f,%f,%d,%d,%d\n", x, y, z, r, g, b);
        }

        fclose(f);
    }

    return 0;
}

I didn't know in which byte order RGB is stored, so you might have to swap R and B. It's usually either RGB or BGR.

Then I used your code to plot the points (I changed read.table to read.csv):

library(rgl)
pcd <- read.csv(file.choose())
names(pcd) <- c("x","y","z","r","g","b")
plot3d(pcd$x,pcd$y,pcd$z,col=rgb(pcd$r,pcd$g,pcd$b,maxColorValue=255))

And this is what I get:

So I'm assuming the problem is with the code where you read your color from the pcd file. The rest looks fine to me.

Update: Your problem is the double type. Change it to float and it should work. Though storing unsigned int as a float is, at the very least, questionable. This is fragile and doesn't not guarantee the colors would be correct after you read them. Some bits might be off.

Another note: you could use >> stream operator to extract numbers from the file. It's much easier than manually parsing it with string methods. You can read about it, for example, here.