How to solve duplicate content for search engines? [closed]

Question

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.

---

Want to improve this question? Update the question so it's on-topic for Stack Overflow.

Closed 7 years ago.

Improve this question

I have 2 websites: Site1.com and Site2.com.

Site1.com has 2 pages: a.htm and b.htm. Site2.com has 3 pages: a.htm, b.htm and c.htm.

Pages a.htm and b.htm are duplication at the 2 sites.

I want search engines to index a.htm and b.htm from Site1 and c.htm from Site2. How and where do I specify this preference? I prefer not to have to specify my preference at the page level (meaning having to edit every page) if possible.

Thanks.

Answer 1

Create a robots.txt file on the root of each server, specifying which pages should be indexed.

For example, on Site 2:

User-agent: *
Disallow: /a.htm
Disallow: /b.htm

Answer 2

You can't directly - no search engine offers that kind of "fill in this form" type of control.

Your two options:

1) use robots.txt to tell crawlers to NOT index the pages on site1
2) use 301 "permanently moved" redirects to tell crawlers to go look at site2 instead of site1.