SimpleOpenNI / Kinect: sceneMap / depthMap Color Control

Question

[Context] I am helping a graduate student (MFA) with an art project:

I am trying to control the output color of the depthMap generation, so that the Red, Green, & Blue values can be independently controlled. The desired end game is that each depthMap generated value can be changed to various shades of Magenta (255,0,255), so that each SimpleOpenNI User will appear differentiated.

Thus far, I have not found a way to do this in the pde file, nor the correct way to modify the SimpleOpenNI class to accomodate the same goal. - For whatever reason, the tint() call changes the alpha value of the canvas, but does not seem to adjust the color values.

Any applicable advice would be appreciated.

Resources: SimpleOpenNI Souce My pde:

import SimpleOpenNI.*;


SimpleOpenNI  context;

void setup()
{
  context = new SimpleOpenNI(this);

  // enable depthMap generation
  if(context.enableScene() == false)
  {
     println("Can't open the sceneMap, maybe the camera is not connected!"); 
     exit();
     return;
  }

  frameRate(30);
  background(128);
  size(context.sceneWidth() , context.sceneHeight()); 
}

void draw()
{

  // update the cam
  context.update(); 

  // draw irImageMap && Sets frame size.
  image(context.sceneImage(),25,25, 590,430);

  // "blend" shapes BG & previous the frame
   tint(#ED145B, 125);

}

Answer 1

Aside from sceneImage(), SimpleOpenNI also provides a method called sceneMap() which takes an int[] as input. It fills the values in the array that map to a particular user with the number corresponding to that user.

For example if you have no users, the sceneMap array will filled with zeroes(e.g.[0,0,0.....0]), if you have 1 user, some elements will be filled with 1 for the pixels belong to that user (e.g. [0,0,0,1,1,...1,0,0,0,1,0...etc.]) and so on.

You could try to do some color replacement in the PImage, but that's not very flexible (what if you have the depth map enabled also ?) and you're rely on the hard coded values for each user(R,B,G,C,M,Y, which ever the order might be in SimpleOpenNI), so the sceneMap() is the more flexible option.

Here's a commented example on how to use this:

import SimpleOpenNI.*;

SimpleOpenNI  context;

int[] sceneMap;//this will store data about the scene (bg pixels will be 0, and if there are any users, they will have the value of the user id - e.g. if there are no users, the array will be filled with zeros, if there is one user, some array entries will be equal to 1, etc. the size of the array is the same as the number of pixels in scene image, so it's easy to use with the pixels[] of a PImage
PImage myUserImage;//this is where we'll draw the user
int user1Colour = color(180,130,30);//change to whatever you like

void setup()
{
  context = new SimpleOpenNI(this);
  context.enableScene();

  background(200,0,0);
  size(context.sceneWidth() , context.sceneHeight());
  //set scene map array
  sceneMap = new int[context.sceneWidth()*context.sceneHeight()];
  //create the image to draw the user into, by default it will be filled black
  myUserImage = createImage( context.sceneWidth() , context.sceneHeight(), RGB );
}

void draw()
{
  context.update(); 

  // // gives you a label map, 0 = no person, 0+n = person n - tell OpenNI to update the numbers in the array
  context.sceneMap(sceneMap);
  //clear myUserImage - fill everything with black
  Arrays.fill(myUserImage.pixels,color(0));//We've never used Arrays.fill() before, but all it does is it loops through all elements on an array you pass and sets a value you want for each element - fills an array with a value
  for(int i = 0 ; i < myUserImage.pixels.length; i++){
    //check if there is a user for the current pixel, if so, use our custom colour for the pixel at this index
    if(sceneMap[i] > 0) myUserImage.pixels[i] = user1Colour;
  }
  myUserImage.updatePixels();

  //display image
  image(myUserImage,0,0);
}

In the long run, if you want to cater for multiple users, it's worth writing a basic utility class:

import SimpleOpenNI.*;

SimpleOpenNI  context;

SceneMapper sceneMap;

void setup()
{
  context = new SimpleOpenNI(this);
  context.enableScene();

  background(200,0,0);
  size(context.sceneWidth() , context.sceneHeight());

  sceneMap = new SceneMapper(context);
}

void draw()
{
  context.update(); 
  sceneMap.update();
  //display image
  image(sceneMap.scene,0,0);
}
class SceneMapper{
  PImage scene;//this is a PImage where we'll actually draw the user with what colour we want
  int[] sceneMap;//this will store scene data - an array of ints which has the same length as context.sceneImage().pixels, the only difference is, sceneImage already has colours set, while scene map has numbers representing user(1,2,3etc.) on top of background(0s)
  int numPixels;//total number of pixels, we only store it so we can reuse it
  color bg;//background colour
  color[] users = {color(255),color(192),color(127),color(64),color(32)};//fill colours for users

  SceneMapper(SimpleOpenNI context){
    numPixels = context.sceneWidth()*context.sceneHeight();
    sceneMap = new int[numPixels];//init scene nap array
    scene = createImage( context.sceneWidth(), context.sceneHeight(), RGB );//create a PImage to display scene data
    scene.loadPixels();  
    bg = color(0);
  }
  void update(){
    context.sceneMap(sceneMap);//ask SimpleOpenNI to store scene map data into our array
    Arrays.fill(scene.pixels, bg);//clear the image - fill it with the background colours
    for(int i = 0 ; i < numPixels ; i++){//loop through all pixels
      for(int u = 0 ; u < users.length; u++){//loop through user colours
        if(sceneMap[i] > 0) scene.pixels[i] = users[u];//if there are user pixels, use set user colour for those pixels (e.g. pixels with value 1 will use colours stored in users[0], pixels with value 2 will use colour from users[1], etc.) 
      } 
    }
    scene.updatePixels();//we've use pixels, so update the image at the end
  }
}

but it depends if the student is familiar with classes, and given it's an MFA, there's less pressure on clean code and more pressure on getting things done I presume.