Question

I'm testing my site with the software called Search Engine Optimization (SEO) Toolkit 1.0 it displays this error :

The content type for the response from "htpp://mysite.com/robots.txt" is "text/html". The Web server should return "text/plain" as the content type for a Robots.txt file.

My robots.txt file is simply this :

User-agent: * 
Allow: /

Saved with UTF-8 Without BOM Encoding.

Is this wrong?

What should be a default, harmless robots.txt file?

Thanks !

Was it helpful?

Solution

This is a MIME type issue and needs to be configured in your server.

Here is a link: http://www.nextthing.org/archives/2007/03/12/robotstxt-adventure

For your specific hosting provider, they insert a small snippet of tracking javascript. Disable that feature following the customer service support in the comments and the mimetype should render.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top