LDAP has no aggregate function to determine the number of members. Some LDAP implementations may have added features for aggregation, but AFIK, Microsoft Active Directory does not.
You could move you baseDN to a higher point to encompass all the possible OUs in which there are groups or even root. As you tagged the question as Microsoft Active Directory, you may then need to chase referrals.
I was not able to determine if Microsoft Active Directory supports extensible matching for DNs which would allow matching only within two or more containers. If Microsoft Active Directory does, then a filter similar to: (&(|(ou:dn:=groups)(ou:dn:=groups2))(objectclass=groups)) might work.
-jim