Solr segment mearge policy configuration

Question

I am using solrcloud 4.0 and i did not change any settings for segment merging (its default to 32 MB). Though my index size is only 18 MB it displays 5 segments on admin UI. I dont understand is it correct or some bug ?

Answer 1

1 segment is created on every commit, then the merge policy is checked, whether a merge is required. Either you made 4 commits or you made more, than the watermark of the merge policy specified, and some segments were merged. You can see the visualization of merges with different merge settings here.

Answer 2

The number of segments in an index is fixed once the index is fully built, but it varies while indexing is in progress. Lucene adds segments as new documents are added to the index, and merges segments every so often.

When new documents are added to a Lucene index, they are initially stored in memory instead of being immediately written to the disk.

For instance, if we set mergeFactor to 10, a new segment will be created on the disk for every 10 documents added to the index. When the 10th segment of size 10 is added, all 10 will be merged into a single segment of size 100. When 10 such segments of size 100 have been added, they will be merged into a single segment containing 1000 documents, and so on. Therefore, at any time, there will be no more than 9 segments in each power of 10 index size.

ramBufferSizeMB

Once accumulated document updates exceed this much memory space (defined in megabytes), then the pending updates are flushed. This can also create new segments or trigger a merge. Using this setting is generally preferable to maxBufferedDocs. If both maxBufferedDocs and ramBufferSizeMB are set in solrconfig.xml, then a flush will occur when either limit is reached.

 <ramBufferSizeMB>32</ramBufferSizeMB>

References:

http://www.onjava.com/pub/a/onjava/2003/03/05/lucene.html

http://docs.lucidworks.com/display/solr/IndexConfig+in+SolrConfig