you can use GCM for that which pushes updates to phone as soon as they happen on server. you can load the contacts at login, but as soon as something changes or created on server you can inform the user about it and can only load that Contact.
check link how to implement it