Grails searchable plugin reindex performance tuning

Question

I need to re-index regulary all of my domain instances from a live legacy database Where are ~400k recodrs in DB Reading domain object one by one from DB withoud reindex getting ~15 minuts

The iterating with reindex gets 8-10 hours with index in remdisk:

ids.each { id ->
      Domain.reindex(id)
}

I can't use Domain.reindex() becouse of DB connections timeout

searchableService.reindexAll(list) take about 2 hours. But this method is deprecated

The index updated by one thread. The result size of index on disk ~0.5Gb

How can I iprove the perfomance on lucena? As I understand, reindex of my info should get about one hour.

Answer 1

According to https://stackoverflow.com/a/13128140/166062 the good settings are:

compassSettings = [
  'compass.engine.optimizer.schedule.period': '300',
  'compass.engine.mergeFactor':'1000',
  'compass.engine.maxBufferedDocs':'1000',
  'compass.engine.ramBufferSize': '128',
  'compass.engine.useCompoundFile': 'false',
  'compass.transaction.processor': 'read_committed',
  'compass.transaction.processor.read_committed.concurrentOperations': 'false',
  'compass.transaction.lockTimeout': '30',
  'compass.transaction.lockPollInterval': '500',
  'compass.transaction.readCommitted.translog.connection': 'ram://'
]