I have found the answer. The reason why I need to elevate privileges for the application is due to the application opening the All User's profile phone book. This is indicated by my call to book.Open()
If you encountered a problem like this remember you can always store the phone book next to your application, which will remove the need to elevate permissions.
The code should be :
RasPhoneBook book = new RasPhoneBook();
book.Open(".\\MyAppPhoneBook.pbk");