Lucene returning inconsistent results counts

Question

In our website we use Lucene queries to get search results. The problem is we have been randomly getting a subset of the correct results sometimes. Usually everything is fine. But when you are doing some things like adjusting search criteria, changing sort order etc, suddenly you see a subset of the results (sometimes 0 results). If you change the sort order again, you get the correct results again and you might never see the issue again.

Have anybody experience this issue and have an idea on what might be causing this? Could it be an index corruption or could it be the high load on the index?

We are using Lucene.Net 2.9.4.1. This is a Sitecore.NET 6.6.0 (rev. 130404) project. Our index has only less than 10000 documents.

Here's the code sample we are using to get total result counts:

        Index index = SearchManager.GetIndex("indexname");

        using (IndexSearchContext searchContext = index.CreateSearchContext())
        {
            Lucene.Net.Search.IndexSearcher searcher = searchContext.Searcher;

            TopScoreDocCollector collector = TopScoreDocCollector.create(1, false);
            searcher.Search(query, collector);
            return collector.GetTotalHits();
        }

We've also had this weird problem in the past. That was solved with a patch provided by sitecore to detect this and reset the Lucene internal variable. Not sure whether these are related.

Answer 1

It seems that the issue was not in the index. But in a static variable we had used to store and display the counts. When the load was high the shared variable got messed up by parallel requests which led to weird results.