Shared Compass/Lucene Index in JDBC Store

Question

Using the searchable plugin in Grails (which uses Compass/Lucene under the hood) we're trying to share a search index between two different web applications. One application accesses the data only in a read-only fashion. The other application allows to modify the data and is in charge of updating the index on any change or do a full re-index on demand.

To store the index we're using the JDBC Store (with both applications pointing to the same DB) http://www.compass-project.org/docs/latest/reference/html/core-connection.html.

Unfortunately, as soon as we rebuild the whole index in one application, the other application seems to have invalid data cached and an exception is thrown if a search is performed:

| Error 2012-05-30 09:22:07,560 [http-bio-8080-exec-8] ERROR errors.GrailsExceptionResolver  - IndexOutOfBoundsException occurred when processing request: [POST] /search
Index: 45, Size: 13. Stacktrace follows:
Message: Index: 45, Size: 13
   Line | Method
->> 547 | RangeCheck  in java.util.ArrayList
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
|   322 | get         in     ''
|   265 | fieldInfo . in org.apache.lucene.index.FieldInfos
|   254 | fieldName   in     ''
|    86 | read . . .  in org.apache.lucene.index.TermBuffer
|   127 | next        in org.apache.lucene.index.SegmentTermEnum
|   158 | scanTo . .  in     ''
|   271 | get         in org.apache.lucene.index.TermInfosReader
|   332 | terms . . . in     ''
|   717 | terms       in org.apache.lucene.index.SegmentReader
|    93 | generate .  in org.apache.lucene.search.PrefixGenerator
|    58 | getDocIdSet in org.apache.lucene.search.PrefixFilter
|   116 | <init> . .  in org.apache.lucene.search.ConstantScoreQuery$ConstantScorer
|    81 | scorer      in org.apache.lucene.search.ConstantScoreQuery$ConstantWeight
|   230 | scorer . .  in org.apache.lucene.search.BooleanQuery$BooleanWeight
|   131 | search      in org.apache.lucene.search.IndexSearcher
|   112 | search . .  in     ''
|   204 | search      in org.apache.lucene.search.MultiSearcher
|   113 | getMoreDocs in org.apache.lucene.search.Hits
|    90 | <init>      in     ''
|    61 | search . .  in org.apache.lucene.search.Searcher
|   146 | findByQuery in org.compass.core.lucene.engine.transaction.support.AbstractTransactionProcessor
|   259 | doFind . .  in org.compass.core.lucene.engine.transaction.readcommitted.ReadCommittedTransactionProcessor
|   246 | find        in org.compass.core.lucene.engine.transaction.support.AbstractConcurrentTransactionProcessor
|   352 | find . . .  in org.compass.core.lucene.engine.LuceneSearchEngine
|   188 | hits        in org.compass.core.lucene.engine.LuceneSearchEngineQuery
|   199 | hits . . .  in org.compass.core.impl.DefaultCompassQuery
|   104 | doInCompass in grails.plugin.searchable.internal.compass.search.DefaultSearchMethod$SearchCompassCallback
|   133 | execute . . in org.compass.core.CompassTemplate
|    57 | doInCompass in grails.plugin.searchable.internal.compass.support.AbstractSearchableMethod
|    66 | invoke . .  in grails.plugin.searchable.internal.compass.search.DefaultSearchMethod
|    37 | search      in grails.plugin.searchable.SearchableService

We could communicate the fact that the index is rebuilt from one to the other application so that some clean-up could be performed.

• Did anybody have a similar problem with Grails and the Searchable plugin?
• Is it possible to discard data cached by Compass/Lucene?
• Is it possible to disable caching generally?

Answer 1

Clearing all caches before searching seems to solve the issue...

searchableService.compass.compass.searchEngineFactory.indexManager.clearCache()