Match 3D point cloud to CAD model

Question

I have a point cloud of an object, obtained with a laser scanner, and a CAD surface model of that object.

How can I match the point cloud to the surface, to obtain the translation and rotation between cloud and model?

I suppose I could sample the surface and try the Iterative Closest Point (ICP) algorithm to match the resulting sampled point cloud to the scanner point cloud.

Would that actually work?

And are there better algorithms for this task?

Answer 1

Yes, ICP can be applied to this problem, as you suggest with sampling the surface. It would be best if you have all available faces in your laser scan otherwise you may have to remove invisible faces from your model (depending on how many of these there are).

One way of automatically preparing a model by getting rid of some of the hidden faces is to calculate the concave hull which can be used to discard hidden faces (which are for example faces that are not close to the concave hull). Depending on how involved the model is this may or may not be necessary.

ICP works well if given a good initial guess because it ignores points that are not close with respect to the current guess. If ICP is not coming up with a good alignment you may try it with multiple random restarts to try and fix this problem, choosing the best alignment.

A more involved solution is to do local feature matching. You sample and calculate an invariant descriptor like SHOT or FPFH. You find the best matches, reject non-consistent matches, use them to come up with a good initial alignment and then refine with ICP. But you may not need this step depending on how robust and fast the random-restart ICP is.

Answer 2

In new OpenCV, I have implemented a surface matching module to match a 3D model to a 3D scene. No initial pose is required and the detection process is fully automatic. The model also involves an ICP.

To get an idea, please check that out a video here (though it is not generated by the implementation in OpenCV):

https://www.youtube.com/watch?v=uFnqLFznuZU

The full source code is here and the documentation is here.

You mentioned that you needed to sample your CAD model. This is correct and we have given a sampling algorithm suited for point pair feature matching, such as the one implemented in OpenCV:

Birdal, Tolga, and Slobodan Ilic. A point sampling algorithm for 3D matching of irregular geometries. 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017.

http://campar.in.tum.de/pub/tbirdal2017iros/tbirdal2017iros.pdf

Answer 3

There's an open source library for point cloud algorithms which implements registration against other point clouds. May be you can try some of their methods to see if any fit.

As a starter, if they don't have anything specific to fit against a polygon mesh, you can treat the mesh vertices as another point cloud and fit your point cloud against it. This is something that they definitely support.