We finally found a workaround here.
We are still using Media table to get latest modified photos, but to fetch them from RMLS/Matrix server, we are using Property resource -
$rets->GetObject("Property", $photoType, $propertyId, $photoNum);
All the parameters such as $propertyId and $photoNum are already available in in Media table, and we know all three photo types as well.
This has fixed our problem of getting several '20403 = No Object Found' errors. Thought of adding it here so that it might help others.