Stream Leap Motion Actual Camera Feed

Question

Is there a way to stream (in JS preferably, but any language would do) the actual infrared camera video feed from the Leap Motion? The demo seen at 0:52 here seems to show that the device can provide more data than just a skeleton of points, and I'd love to be able to display the actual "Leap-View" data in one of my projects, which I would assume would essentially be a grayscale image.

Thanks!

Answer 1

My name's Edwin with the Leap Motion Community Team. Unfortunately the "point clouds" that were featured in our early videos are visualizations from some of our debugging tools. Because they’re not temporally or spatially consistent, they are not usable as methods of interaction. There is currently no point cloud to be had. It may be something we can reconstruct from the 3D information we do have, but probably not a feature we will add in the short term.

Answer 2

I think what you want is this: https://github.com/meyburgh/forirony/blob/master/misc/leap.cpp

Which is a very simple demo that shows the gray scale infrared video from each of the leap motion's cameras.

The video looks a bit strange, so if you want it to look 'normal' you need to rectify it--Leap provides image.rectify(), but that occurs on the cpu, so for performance it's best to use a shader instead of the image.rectify() function.

To get the 'point cloud,' if that's what you're interested in, you could do per pixel disparity mapping (which opencv has on the cpu/gpu) or you can check out NVIDIA's CUDA toolkit which has a disparity map demo included in the samples. link to opencv's stereo correspondence (aka disparity mapping): http://docs.opencv.org/3.0-beta/modules/cudastereo/doc/stereo.html

I can appreciate that the quality of the point cloud through disparity mapping would be quite coarse and noisy, and thus not useful for 'interaction' as Edwin put it in his post, but if you are interested in studying statistical techniques to make sense of the information hidden in the noise, or would like the point cloud for 'artistic' reasons then this is the way to go I would say.