Since I need read access only, I decided to use jsonp to access the contacts. Otherwise, I need to use an additional library which is deprecated and a deprecated api version (because the gdata client library doesn't support the contacts api version 3).
Since I have the access token already, it's pretty easy to get contact data:
http.jsonp(https://www.google.com/m8/feeds/contacts/someone@gmail.com/full?access_token=xyz&alt=json&callback=JSON_CALLBACK")
.success(function(data, status, headers, config) { ...})
.error(function(data, status, headers, config) {...})