This is no different than a user manually entering the IP in the address bar and making a request.
If anything, you don't have to worry about the DoS or some other vulnerability, the IP owner is the one receiving the request.
That said. Most webservers don't respond to IP-only HTTP requests, you need the hostname in there.