Ok I figured it out myself. The ROI in the actual image can be cropped so as to get a new image. Comparing the actual object size and the ROI size, if the ratio is beyond a specific value, we can determine that the object is lost.
In my case, the ratio is taken as 0.3