add block urls in robots.txt rule

Question

I want to allow only these urls to read robots in mysite

example.com/site/faq
example.com/site/blog
example.com/site/aboutus

All other URLs need to be blocked, what are the rules ? Thanks

Answer 1

I think what you're looking for is:

User-agent: *
Allow: /site/faq
Allow: /site/blog
Allow: /site/aboutus
Disallow: *

That specifically allows the three folders you mentioned, and disallows everything else.