Fixed it!
The issue was I did not include the path in the string, although I am not sure why the path is needed, I thought it would use the default...
document.cookie = "loggedIn=; path=/; expires=Thu, 01 Jan 1970 00:00:00 GMT";
The above works!