Googlebot indexing my localhost dev machine - where did it find my IP?

Question

I've discovered Googlebot indexing my development site (home PC) via its IP address. Surprise. I've changed my .htaccess file to prevent future access, but...

How did Googlebot find me anyway? I made a request to Google to index my live site, but there shouldn't be any links to my IP anywhere on the web.

The only place my IP is listed on my site is in a PHP function that is used to exclude my address from being logged. Can Googlebot (or any bot) harvest IP addresses from raw PHP code?

Answer 1

IP addresses can't likely be harvested from your PHP code because the web server will execute the PHP script and only send the result to the browser.

But there are lots of bots that just scan random IP addresses on port 80 and look for vulnerable software, often using Google's user agent string - did you check if the request's IP address actually belongs to Google? There is even a search engine for IP addresses that have web servers running, you could check if you can find your own host: http://www.shodanhq.com

It is generally a bad idea to let a development server listen on 0.0.0.0, i.e. expose it to the Internet. If you don't need to access it from the outside, let it listen on 127.0.0.1 or you could run into trouble if you don't update it often.