SPFx: inform users when Search last indexed and when Items will appear in results

Question

Scenario: Large team processes a high volume of customer data (multiple Libraries, Document Sets and Sites/ Sites Collections) Part of it is trading information so accurate info is vital.

I have an SPFx webpart displaying Search results,
but (of course) non-indexed Items do not show (yet)

• Is information about the last crawl available? (Client side tech only, so no Powershell)

• Can a (gu)estimate be made about when the Items will show up in Search Results?
  (next crawl time)

Indexing ALL information myself is impossible (because of the amount) so I am after a mechanism where software or user can make a decision to go get part of the information

Ideally I want to know when (the contents of) a Document Set/Folder/Library was last indexed and then force a re-index or dig into Libraries myself.

Answers I found point to PowerShell, I need this Client Side from SPFx

sources:

• https://support.office.com/en-us/article/Manage-the-search-schema-in-SharePoint-Online-d4fab46d-ba41-4c03-9d4c-32b5b33198b6?ui=en-US&rs=en-US&ad=US&fromAR=1
• http://www.rapidcircle.com/crawling-as-fast-as-possible/

Answer 1

It used to available some time back using the managed property Crawltime, however in recent tenants that i have used, this is no longer the case. However, Mikael Svenson has managed to find a great way to retrieve the crawl time of items.

You can follow the steps mentioned below:

1) Save the below mentioned xml configuration in xml file.

2) Go to the SharePoint admin center > Search > Import search configuration.

3) Import the xml file

4) It will now map the crawled property Internal:323 to RefinableString100 which will give you the crawled time of the item.

<SearchConfigurationSettings xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/Microsoft.Office.Server.Search.Portability">
    <SearchQueryConfigurationSettings>
        <SearchQueryConfigurationSettings>
            <BestBets xmlns:d4p1="http://www.microsoft.com/sharepoint/search/KnownTypes/2008/08" />
            <DefaultSourceId>00000000-0000-0000-0000-000000000000</DefaultSourceId>
            <DefaultSourceIdSet>true</DefaultSourceIdSet>
            <DeployToParent>false</DeployToParent>
            <DisableInheritanceOnImport>false</DisableInheritanceOnImport>
            <QueryRuleGroups xmlns:d4p1="http://www.microsoft.com/sharepoint/search/KnownTypes/2008/08" />
            <QueryRules xmlns:d4p1="http://www.microsoft.com/sharepoint/search/KnownTypes/2008/08" />
            <ResultTypes xmlns:d4p1="http://schemas.datacontract.org/2004/07/Microsoft.Office.Server.Search.Administration" />              
            <Sources xmlns:d4p1="http://schemas.datacontract.org/2004/07/Microsoft.Office.Server.Search.Administration.Query" />
            <UserSegments xmlns:d4p1="http://www.microsoft.com/sharepoint/search/KnownTypes/2008/08" />
        </SearchQueryConfigurationSettings>
    </SearchQueryConfigurationSettings>
    <SearchRankingModelConfigurationSettings>
        <RankingModels xmlns:d3p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays" />
    </SearchRankingModelConfigurationSettings>
    <SearchSchemaConfigurationSettings>
        <Aliases xmlns:d3p1="http://schemas.datacontract.org/2004/07/Microsoft.Office.Server.Search.Administration">
            <d3p1:LastItemName>PzlCrawlTime</d3p1:LastItemName>
            <d3p1:dictionary xmlns:d4p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
                <d4p1:KeyValueOfstringAliasInfoy6h3NzC8>
                    <d4p1:Key>PzlCrawlTime</d4p1:Key>
                    <d4p1:Value>
                        <d3p1:Name>PzlCrawlTime</d3p1:Name>
                        <d3p1:ManagedPid>1000000900</d3p1:ManagedPid>
                        <d3p1:SchemaId>80610</d3p1:SchemaId>
                    </d4p1:Value>
                </d4p1:KeyValueOfstringAliasInfoy6h3NzC8>
            </d3p1:dictionary>
        </Aliases>
        <CategoriesAndCrawledProperties xmlns:d3p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
            <d3p1:KeyValueOfguidCrawledPropertyInfoCollectionaSYUqUE_P>
                <d3p1:Key>012357bd-1113-171d-1f25-292bb0b0b0b0</d3p1:Key>
                <d3p1:Value xmlns:d5p1="http://schemas.datacontract.org/2004/07/Microsoft.Office.Server.Search.Administration">
                    <d5p1:LastItemName>323</d5p1:LastItemName>
                    <d5p1:dictionary>
                        <d3p1:KeyValueOfstringCrawledPropertyInfoy6h3NzC8>
                            <d3p1:Key>323</d3p1:Key>
                            <d3p1:Value>
                                <d5p1:Name>323</d5p1:Name>
                                <d5p1:CategoryName>Internal</d5p1:CategoryName>
                                <d5p1:IsImplicit>false</d5p1:IsImplicit>
                                <d5p1:IsMappedToContents>false</d5p1:IsMappedToContents>
                                <d5p1:IsNameEnum>true</d5p1:IsNameEnum>
                                <d5p1:MappedManagedProperties />
                                <d5p1:Propset>012357bd-1113-171d-1f25-292bb0b0b0b0</d5p1:Propset>
                                <d5p1:Samples />
                                <d5p1:SchemaId>80610</d5p1:SchemaId>
                            </d3p1:Value>
                        </d3p1:KeyValueOfstringCrawledPropertyInfoy6h3NzC8>
                    </d5p1:dictionary>
                </d3p1:Value>
            </d3p1:KeyValueOfguidCrawledPropertyInfoCollectionaSYUqUE_P>
        </CategoriesAndCrawledProperties>
        <CrawledProperties xmlns:d3p1="http://schemas.datacontract.org/2004/07/Microsoft.Office.Server.Search.Administration" i:nil="true" />
        <ManagedProperties xmlns:d3p1="http://schemas.datacontract.org/2004/07/Microsoft.Office.Server.Search.Administration">
            <d3p1:LastItemName i:nil="true" />
            <d3p1:dictionary xmlns:d4p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays" />
            <d3p1:TotalCount>0</d3p1:TotalCount>
        </ManagedProperties>
        <Mappings xmlns:d3p1="http://schemas.datacontract.org/2004/07/Microsoft.Office.Server.Search.Administration">
            <d3p1:LastItemName>012357bd-1113-171d-1f25-292bb0b0b0b0:323-&gt;1000000900</d3p1:LastItemName>
            <d3p1:dictionary xmlns:d4p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
                <d4p1:KeyValueOfstringMappingInfoy6h3NzC8>
                    <d4p1:Key>012357bd-1113-171d-1f25-292bb0b0b0b0:323-&gt;1000000900</d4p1:Key>
                    <d4p1:Value>
                        <d3p1:Name>012357bd-1113-171d-1f25-292bb0b0b0b0:323-&gt;1000000900</d3p1:Name>
                        <d3p1:CrawledPropertyName>323</d3p1:CrawledPropertyName>
                        <d3p1:CrawledPropset>012357bd-1113-171d-1f25-292bb0b0b0b0</d3p1:CrawledPropset>
                        <d3p1:ManagedPid>1000000900</d3p1:ManagedPid>
                        <d3p1:MappingOrder>100</d3p1:MappingOrder>
                        <d3p1:SchemaId>80610</d3p1:SchemaId>
                    </d4p1:Value>
                </d4p1:KeyValueOfstringMappingInfoy6h3NzC8>
            </d3p1:dictionary>
        </Mappings>
        <Overrides xmlns:d3p1="http://schemas.datacontract.org/2004/07/Microsoft.Office.Server.Search.Administration">
            <d3p1:LastItemName>1000000600</d3p1:LastItemName>
            <d3p1:dictionary xmlns:d4p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
                <d4p1:KeyValueOfstringOverrideInfoy6h3NzC8>
                    <d4p1:Key>1000000900</d4p1:Key>
                    <d4p1:Value>
                        <d3p1:Name>1000000900</d3p1:Name>
                        <d3p1:AliasesOverridden>true</d3p1:AliasesOverridden>
                        <d3p1:ExtraProperties i:nil="true" />
                        <d3p1:ManagedPid>1000000900</d3p1:ManagedPid>
                        <d3p1:MappingsOverridden>true</d3p1:MappingsOverridden>
                        <d3p1:SchemaId>80610</d3p1:SchemaId>
                        <d3p1:TokenNormalization>true</d3p1:TokenNormalization>
                    </d4p1:Value>
                </d4p1:KeyValueOfstringOverrideInfoy6h3NzC8>
            </d3p1:dictionary>
        </Overrides>
    </SearchSchemaConfigurationSettings>
</SearchConfigurationSettings>

References -

Crawl time for SPO and 2013

Crawl time xml reference - latest xml configuration

In my tenant, it got mapped to RefinableString100. So after that, I can use this Refinable anywhere in REST api query or csom code etc.

So i guess this answers your first question

Is information about the last crawl available? (Client side tech only, so no Powershell)

You can use SPFx to do a search REST call to fetch the last crawl time and display it in the SPFx webpart UI.

Regarding the second question

Can a (gu)estimate be made about when the Items will show up in Search Results? (next crawl time)

Based on the results given in the first answer, you might need to store that value somewhere , a SP custom list(i guess) and update it. Then next time whenever the crawl occurs again, the value will be overridden.Then you need to architect the date difference mechanism. I think versioning would be one option where we might be able to get date & time difference between two versions.