prevent google to index dynamic error pages (none 404) [closed]

Question

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.

---

This question does not appear to be about programming within the scope defined in the help center.

Closed 8 years ago.

Improve this question

There are some none 404 error pages on my website. what is the best way to stop google from indexing them?

option 1

header("HTTP/1.0 410 Gone");

what if the content is not gone? for example: the article does not exist. or wrong parameter has been caught

option 2

<meta name="robots" content="noindex" />

does it only affect one page or the whole domain?

option 3

using 404 which will make some other problems and I would like to avoid.

robot.txt

this option will not work since the error will depend on the database and is not static.

Answer 1

Best practice is to do a 301 redirect to similar content on your site if content is removed.

To stop Google indexing certain areas of your site use robots.txt

UPDATE: If you send a 200 OK and add the robots meta tag (Option 2 in your question) - this should do what you want.

Answer 2

One way to prevent google bots to index something is using robots files:

User-agent: googlebot
Disallow: /mypage.html
Disallow: /mp3/

This way you can manually disable single pages or entire directories.