Steps to detect and track droplets in a video using OpenCV C++

Question

I am currently doing a project where I need to locate ink drops in a video, perform measurements such as volume estimation, velocity, and distance travelled before it becomes spherical.

Firstly, I would like to know whether I am along the right tracks in tackling this project. At the moment I have:

1.) Converted the original image to grayscale

2.) Applied Gaussian Blur then Canny edge detection (Click here for image)

3.) Located the white pixels using findNonZero() then calculated the sum of blocks of rows and the block with the highest concentration of white pixels and all the rows above it are cropped out). This removes the print heads in the image so the ROI is only the droplets below it.

4.) Used the findContours to find the contours. (Click here for image)

The above 4 steps are what I have done so far. Are the following steps below what I should do next?

• Dilate the binary image first after cropping and before finding contours to ensure the contours will be closed and not open?
• Maybe ignore the ones that are very open? (Any tips on how to actually do this?)
• floodFill() every closed circles
• Find the each contours' area using contourArea() (Can I then estimate the volume of the drop after this step with a few assumptions like its shape, pixel to volume ratio, etc?)
• Find the centre of each contour and save it to an array so I can compare it to the centre of the same drop in the next frame. Once I know distance travelled of the centre of the droplet and the frame rate of the video I should be able to estimate velocity.

I am also unsure of how I can give a drop an ID so I can be sure I am tracking it properly and know when a new drop has entered the ROI.

Any help would be greatly appreciated, Thank You.

Answer 1

I think that your idea is good and can be quite easily extended to something that will satisfy you.

For clarification i will call red ROI from your image "redROI".

1. Find all droplets in redROI. Remember positions and IDs.
2. For each droplet position from previous step create a ROI similar to yellow rectangle: For each rectangle check whether there is a droplet inside it.
   If yes - probably it's the droplet from the previous frame, so the one you are looking for.
   If no - you may try to search again in a little bit bigger rectangle or assume that the darkset point of this ROI is you droplet. If ROI is near bottom of redROI probably the droplet is gone - forget about it.

Note few things:
-size of the rectangle depends on how fast droplets move and whether they can move only vertical or diagonal (wind can change direction of move) too.
-before searching for droplets, check whether all rectangles are disjoint (the don't have any common area -> (Rect1 & Rect2).area() == 0 for each pair of rectangles).
-before searching for droplets in ROI make sure this ROI is inside redROI. So just use this code: roi = roi & redROI;
After finding new positions of every old droplet, search for droplets in whole redROI, so you won't miss any new droplet.

Let me know if you don't understand some part of this idea - i will try to explain it better.

Maybe ignore the ones that are very open? (Any tips on how to actually do this?)

I'm not sure about it, so check it. Try to use CV_RETR_LIST as the third parameter of findContours and check the distance between first and the last point from returned (by findContours) contour - if the distance is big than the contour is open, if no - it is closed.

floodFill() every closed circles

You can just use drawContours and set thickness parameter to -1 - simpler and faster solution.

edit:
You can try to use optical flow as well - it's already implemented in openCV, here you can read nice tutorial about that: http://robotics.stanford.edu/~dstavens/cs223b/ (start from .pdf files)