How to merge Two data sets of point clouds or polygons into one? (Merging not Appending)

Question

I am developing a real-time scanner, in which I would be able to scan the surface in real-time. Untill now I can scan one patch of the surface and can save it. One patch means just 1 scan of the surface (point cloud or triangles).

What I want is, I want to scan multiple patches in real-time. For this purpose, I have to merge the previous patch with the current patch. But I don't know what is the standard way or algorithm to merge two patches and also I don't know which one is the best way for merging, for example, before triangulation (point cloud merging), or after the triangulation (mesh merging). Merging means remove overlapped points or triangles

My idea: If there are two point clouds, source and target then using VTK, find a closest point from the target point cloud, and select one point the other is discarded. Is this a method for merging ? This is just my idea? But the problem is, the number of points in source and target will be different.

How can I merge two patches using VTK, kindly guide me?

Also suggest me what is the standard and optimum way to achieve real-time scanning task.

Case # 1:
i) Point Cloud Aquisition

ii) Register

iii) Merge

iv) Triangulate

Case # 2:

i) Point Cloud Aquisition

ii) Register

iii) Triangulate

iv) Merge

Case # 3:

i) Point Cloud Aquisition

ii) Triangulate

iii)Register

iv) Merge

Please guide me. Thanks.

Answer 1

I put a note here because I have just been thinking about something similar.

Your suggested method (doing a nearest neighbour search during the merge) does seem possible. The issue about there being different sizes between the two clouds being merged does not seem to be an issue if you do a radius search based on some desirable resolution rather than a search for 1 neighbour.

To manage your case 1, you could try merging all the clouds then downsampling with a voxel grid e.g. pcl::VoxelGrid before the triangulation (this would be the easiest way but may not be what you want).

The algorithm encapsulated in pcl::GreedyProjectionTriangulation seems to be mostly described in the below paper [1]. In that paper they also describe an incremental mesh update procedure which is a minor change to the algorithm (they remove triangles close to a new point and start the greedy triangulation again). As far as I know, this has not been implemented in PCL but shouldn't be too difficult. This would correspond to your case 2. However, the mesh you get out would depend on the order in which you merged the clouds. Because it is a time investment I would suggest trying the point-based merging first.

[1] Marton, Z. C., R. B. Rusu, and M. Beetz. 2009. “On Fast Surface Reconstruction Methods for Large and Noisy Point Clouds.” In IEEE International Conference On Robotics and Automation, 3218–3223. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5152628.