Question
is there any point in having a robots.txt with just one line of content like below
User-agent: *
compared to not having it at all ?
Solution
No, google (and let's not beat around the bush, Google is the search engine you should care about) by default will crawl everything unless you tell it not to.
If you want to help Google, use google webmaster tools and submit a sitemap in the correct format
http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156184
OTHER TIPS
No; there is no point in doing that.
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow