I don`t know either. Distance is measure of the feature point similarity, less is better. The original ORB paper (fig. 5, below) shows distribution of the distances for good and bad matches. One can surely says that "good" distance threshold would be around 64.
So more correct is :
double dist_th = 64;
for( int i = 0; i < descriptors_object.rows; i++ )
{ if( matches[i].distance < dist_th )
{ good_matches.push_back( matches[i]); }
}
And then you still have to use RANSAC to filter inconsistent matches. So, the simplest solution is to do match you query image with all 4 database images.
But I`d advise you to use some classifier, not just matching. See this guy approach (it works, I know him) - http://cmp.felk.cvut.cz/~sulcmila/