Upgrading sitecore 6.6 index configuration to sitecore 7 (using ComputedFields)

Question

Sitecore CMS+DMS 6.6.0 rev.130404 => 7.0 rev.130424

In our project we have been using AdvancedDatabaseCrawler (ADC) for our indexes (specially because of it's dynamic fields feature). Here's a sample index configuration:

<index id="GeoIndex" type="Sitecore.Search.Index, Sitecore.Kernel">
<param desc="name">$(id)</param>
<param desc="folder">$(id)</param>
<analyzer ref="search/analyzer" />
<locations hint="list:AddCrawler">
  <web type="scSearchContrib.Crawler.Crawlers.AdvancedDatabaseCrawler, scSearchContrib.Crawler">
    <database>web</database>
    <root>/sitecore/content/Globals/Locations</root>
    <IndexAllFields>true</IndexAllFields>
    <include hint="list:IncludeTemplate">
      <!--Suburb Template-->
      <suburb>{FF0D64AA-DCB4-467A-A310-FF905F9393C0}</suburb>
    </include>
    <dynamicFields hint="raw:AddDynamicFields">
      <dynamicField type="OurApp.CustomSearchFields.SearchTextField,OurApp" name="search text" storageType="NO" indexType="TOKENIZED" vectorType="NO" />
      <dynamicField type="OurApp.CustomSearchFields.LongNameField,OurApp" name="display name" storageType="YES" indexType="UN_TOKENIZED" vectorType="NO" />
    </dynamicFields>
  </web>
</locations>
</index>

As you can see, we use scSearchContrib.Crawler.Crawlers.AdvancedDatabaseCrawler as the crawler and it uses the fields defined inside <dynamicFields hint="raw:AddDynamicFields"> section to inject custom fields into the index.

Now we are upgrading our project to sitecore 7. In Sitecore 7, they have ported the DynamicFields functionality from ADC into sitecore. I found out some articles on this and converted our custom search field classes to implement sitecore 7 IComputedIndexField interface instead of inheriting from BaseDynamicField class in ADC. Now my problem is how to change the index configuration to match with new sitecore 7 APIs. There were bits and pieces on the web but couldn't find all the examples I needed to convert my configuration. Can anybody help me on this?

While I'm doing this I'm under the impression that we won't have to rebuild our indexes since it still uses Lucene internally. I don't want to change the index structure. Just want to upgrade the code and configuration from AdvancedDatabaseCrawler to Sitecore 7. Should I be worried about breaking our existing indexes? Please shed some light on this as well.

Thanks

Answer 1

I managed to convert the index configuration for sitecore ContentSearch API. Looking at Sitecore default index configurations was a great help for this.

Note: As also mentioned by Stephen, <include hint="list:IncludeTemplate"> does not work in Sitecore 7.0 initial release. It's fixed in Sitecore 7.0 rev. 131127 (7.0 Update-3) and I'm planning to upgrade to it.

Here's a good article on sitecore 7 index update strategies by John West. It'll help you in configuration your indexes the way you want.

Converted configuration:

<sitecore>
<contentSearch>
  <configuration type="Sitecore.ContentSearch.LuceneProvider.LuceneSearchConfiguration, Sitecore.ContentSearch.LuceneProvider">
    <DefaultIndexConfiguration type="Sitecore.ContentSearch.LuceneProvider.LuceneIndexConfiguration, Sitecore.ContentSearch.LuceneProvider">
      <IndexAllFields>true</IndexAllFields>
      <include hint="list:IncludeTemplate">
        <!--Suburb Template-->
        <suburb>{FF0D64AA-DCB4-467A-A310-FF905F9393C0}</suburb>
      </include>
      <fields hint="raw:AddComputedIndexField">
        <field fieldName="search text" storageType="NO" indexType="TOKENIZED" vectorType="NO">OurApp.CustomSearchFields.SearchTextField,OurApp</field>
        <field fieldName="display name" storageType="YES" indexType="UN_TOKENIZED" vectorType="NO">OurApp.CustomSearchFields.LongNameField,OurApp</field>
      </fields>
    </DefaultIndexConfiguration>
    <indexes hint="list:AddIndex">
      <index id="GeoIndex" type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider">
        <param desc="name">$(id)</param>
        <param desc="folder">$(id)</param>
        <!-- This initializes index property store. Id has to be set to the index id -->
        <param desc="propertyStore" ref="contentSearch/databasePropertyStore" param1="$(id)" />
        <strategies hint="list:AddStrategy">
          <!-- NOTE: order of these is controls the execution order -->
          <strategy ref="contentSearch/indexUpdateStrategies/onPublishEndAsync" />
        </strategies>
        <commitPolicy hint="raw:SetCommitPolicy">
          <policy type="Sitecore.ContentSearch.TimeIntervalCommitPolicy, Sitecore.ContentSearch" />
        </commitPolicy>
        <commitPolicyExecutor hint="raw:SetCommitPolicyExecutor">
          <policyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch" />
        </commitPolicyExecutor>
        <locations hint="list:AddCrawler">
          <crawler type="Sitecore.ContentSearch.LuceneProvider.Crawlers.DefaultCrawler, Sitecore.ContentSearch.LuceneProvider">
            <Database>web</Database>
            <Root>/sitecore/content/Globals/Countries</Root>
          </crawler>
        </locations>
      </index>
    </indexes>
  </configuration>
</contentSearch>
</sitecore>

Answer 2

A few quick clarifications :)

We have not merged ADC into Sitecore 7, the ContentSearch layer is a complete rewrite of the old search layer for Sitecore. We have taken some of the core concepts from ADC, such as dynamic fields, and put them in the new implementation (as ComputedFields). They are not 1:1 compatible and you will have to do some work on your indexes.

The version of Lucene has also been upgraded from 2.* to 3.0.3 so all indexes will need to be re-created anyway as they are a very different version of Lucene.

There are two options here, the old Lucene search (Sitecore.Search namespace) (which ADC was built upon) has not been touched and will still work in the same way, although I am not sure about ADC compatibility with SItecore 7 as in theory this has now been superseded.

The next option is to update your index to take advantage of the new search features of Sitecore 7. The configuration you have will not be directly compatible but, while you will need to rework your index into the new configuration, the basic concepts should be familiar to you. The dynamic fields you have should map logically to ComputedFields (fields that are calculated when an item is indexed) and everything else is straightforward.

While it looks like a lot of extra config for ContentSearch a lot of it is default config that you will not need to touch, you will just need to override the configuration parts for the computed fields you want to add and the template you want to include.

An example of creating your own configuration override can be found here : http://www.mikkelhm.dk/post/2013/10/12/Defining-a-custom-index-in-Sitecore-7-and-utilizing-it.aspx

I would also recommend making sure you upgrade to 7.0 rev. 131127 (7.0 Update-3) as this fixes a bug in the IncludeTemplates logic in the version you currently have.