apparently TFS is real pick on which IIS bindings are configured. By reverting to the default "*" bindings instead of a specific host header I was able to get all the security buttons to work.
Now I have two bindings for the tfs website in iis.
1st: *:8080 2nd: tfs.company.com:8080