I did a quick testing, and the max-results parameter is working as intended.
I have total 6 contact groups. Happy 1, Happy 2, Happy 3, Happy 4, Happy 5 and Happy 6.
This is the request I made:
https://www.google.com/m8/feeds/groups/userEmail/full?start-index=1&max-results=5
In the response, I get the parameter <openSearch:itemsPerPage>5</openSearch:itemsPerPage>
, and it is only showing Happy 1 - 5 in the result.
I think if you are using the oauth_requestor_id, the URL should looks like this:
https://www.google.com/m8/feeds/groups/userEmail/full?xoauth_requestor_id=userEmail&start-index=10&max-results=5