what are the important considerations when using /robots.txt & /sitemap.xml files

Question

is /robots.txt & /sitemap.xml files will be security vulnerabilities for SharePoint Sites?

what are the important considerations when using /robots.txt:

can i remove /robots.txt,/sitemap.xml file from root directory of web application for internet facing sites for save web site from malware robots that scan the web for security vulnerabilities.

what happen when i remove /sitemap.xml file for web application?

Answer 1

These files will not cause security issues as they need manually provision in a SharePoint farm. Removing these files will have no influence on your farm.

You can refer to an article here for introduction: Sitemaps in SharePoint 2013.

Sitemaps basically inform the search engines about the pages and their metadata, present in your site. This help the crawlers in picking up those pages. This is a great feature especially for public facing websites.