Noindex, nofollow - is placing them in HTTP response enough?

StackOverflow https://stackoverflow.com/questions/21456465

题

I think that question's title says it all - while developing and deploying betas, should I put 

X-Robots-Tag: noindex, nofollow

in HTTP response, or

<meta name="robots" content="noindex, nofollow">

in <head></head> section of every page?

有帮助吗？

解决方案

The X-Robots-Tag header and the robots meta tag have exactly the same meaning to any robot that supports them both. The robots meta tag is a much older standard. The X-Robots-Tag header was later introduced as an alternative to robots meta, for webmasters who had control over server config but not over the site code.

The one difference is that the X-Robots-Tag header is not quite as widely supported as the robots meta tag. If you only care about major search engines, either one is fine, but if you need it to work on older web robots, the meta tag is the better bet.

Note that neither of these prevent the robot from reading pages from your site, they just tell the robot not to index anything or follow any links. If you want to stop web robots from requesting any pages at all, you might consider using robots.txt instead.

其他提示

The META Tag should do it pretty well for you. All the big search engines read the meta tag that has the robots name and NOINDEX,NOFOLLOW content.

许可以下： CC-BY-SA 和 归因
不隶属于 StackOverflow
scroll top