I've worked it out.
I switched Visual Studio to running with Administrative Privileges and also found the .exe for IIS Express and set that to always run with administrative privileges. I'm not 100% sure the second was needed but did them both anyway.
I hope this helps some one else who's had this same issue.
EDIT
(19/08/2013)
This only seemed to work in Windows 7, On Windows 8 I eventually just gave up and now use the full blow IIS for all my dev work.
EDIT
(07/06/2018)
@Tarostar (Comment below), got this working on Windows 10 VS2017 and a number of others have got it to work on Windows 8 prior to that using the same method. So possible I was doing something wrong on my Windows 8 box.