How to Prevent Search Returning Welcome Page Result When not Published for Anonymous users

Question

We have a publishing site in SharePoint 2013 using the traditional subsite hierarchical based navigation. When a new subsite is created it inherits permissions and is therefore set to allow anonymous access.

The problem is, despite the fact that the Welcome Page remains "unpublished", it is being displayed in search results for anonymous users. This is presumably due to the fact that the subsite is configured as anonymously accessible, even though the Welcome Page isn't (yet). The search result appears with a "/" path rather than the full path to the Welcome Page.

This is undesirable as content authors will typically create a new subsite and then work on that for a while before publishing all the pages.

Does anyone have any idea if there is a way of preventing this, or if this is just a bug? Essentially we don't want the subsite to show up in anonymous search results until the Welcome Page is published.

I realise we can get content managers to go into the site settings and turn off the search visibility for that site but they then have to remember to go and turn it back on again. They also won't see the pages showing up in search results when authenticated.

Thanks for any help with this.

Answer 1

A few months later I've cracked this myself. It seems there is a "Size" Managed Property available that represents the size of a page/item. So on subsites for which their Welcome Page is unpublished this Size property is set to 0 (zero) (presumably because the page could not be crawled). So in order to prevent unpublished subsites from appearing in the search results I added the following to the associated search query -

NOT(ContentClass:STS_Web AND Size:0)

This effectively means don't return results that are classed as a subsite and have a Size of 0. As soon as you publish the Welcome Page the Size property gets updated to reflect the size of the page and thus starts appearing in the search results. Annoying, but a solution, finally.

Answer 2

I have done some extensive research on this topic a few weeks ago because I had a similar requirement on the project I worked on.
Even though you can setup crawl rules that exclude subsites, you cannot setup conditions to take page X is not published into account.

What I ended up doing was creating a TimerJob which ran a couple of times a day (based on how fast a site should show up in Search) and looped through all subsites, checked which page was set as the landing page and then check in the Pages library if that page was currently Published.

• If it was not published, turn off the search visibility.
• If it was published, turn on the search visibility.

I'm keen to know myself if there exist OOTB behavior to achieve this without custom code.