By the message provided, I assume you are not explicitly telling the user, that you are uploading his contacts. Setting the read agreement only asks the user, to give you permission to read (and only read) his contacts, but does not give you permission to upload his data anywhere.
What google is telling you, is to add a section in your EULA where explicitly note, that the users contacts are uploaded to a service so you can sync data.