Pregunta

I'm testing my site with the software called Search Engine Optimization (SEO) Toolkit 1.0 it displays this error :

The content type for the response from "htpp://mysite.com/robots.txt" is "text/html". The Web server should return "text/plain" as the content type for a Robots.txt file.

My robots.txt file is simply this :

User-agent: * 
Allow: /

Saved with UTF-8 Without BOM Encoding.

Is this wrong?

What should be a default, harmless robots.txt file?

Thanks !

¿Fue útil?

Solución

This is a MIME type issue and needs to be configured in your server.

Here is a link: http://www.nextthing.org/archives/2007/03/12/robotstxt-adventure

For your specific hosting provider, they insert a small snippet of tracking javascript. Disable that feature following the customer service support in the comments and the mimetype should render.

Licenciado bajo: CC-BY-SA con atribución
No afiliado a StackOverflow
scroll top