I have solved the problem for both old androids and new androids. The problem was that i should have used the view "view_raw_contacts" instead of the table "raw_contacts". Works on all androids i have tested.
Updated selection:
String selection = ContactsContract.Contacts._ID + " IN (" + "SELECT " +
ContactsContract.RawContacts.CONTACT_ID + " FROM " +
"view_raw_contacts" + " WHERE " +
ContactsContract.RawContacts.ACCOUNT_TYPE + " IS NULL OR " +
ContactsContract.RawContacts.ACCOUNT_TYPE + " = 'com.google'" + ")";
On the real device i extract the view name with a regex hoping that it would work on all devices, even on those who modified the contact provider.