Dopo un nucleo di scambio (reindex -> live) La mia schiava è corrotta

StackOverflow https://stackoverflow.com//questions/9685016

Domanda

Sto avendo un problema molto strano qui, ho impostato la mia architettura di solr come supposto Qui e tutto funziona bene tranne che la mia macchina slave solr viene corrotta. Il set-up (Test) è il seguente:

SOLR Setup

    .
  • Un master solr con due core (reindex e live)
  • uno slave solr con due core (reindex e live)
  • Pollinterval set su 00:00:02
  • Replica dopo un ottimizzazione o commit
  • Esecuzione SOLR sotto Ubuntu (11.10) versione 1.4.0.

Il processo

L'intero Proces va come questo:

    .
  1. Pulire l'indice sul master reindex core
  2. noi Disabilita la replica sul master per il nucleo reindex Avvia i dati di DATAIstporthandler Importa i dati MySQL in Peices di 10.000 documenti (per prevenire Long MySQL Locks)
  3. AFTHER THE DATAIMPHERTHANDLER è fatto (4 milioni di record) Abilita la replica sul master
  4. Gli slave prenderanno il nuovo core reindex dal master
  5. una volta La recupero è fatto guardando la proprietà "iserlicatoria" Edizione A Swap Core

    6. Supponiamo che il nucleo vivo abbia 5000 documenti e il Reindex Core 20000 dopo una completa ricostruzione.

    Il risultato (errato)

    Dopo lo swap Questo è il risultato:

    Master: Live (20000), Reindex (5000) SLAVE: Live (20000), Reindex (20000)

    Sembra che lo slave copia il suo nucleo reindex al nucleo vivo ma non scambia il nucleo vivo con il nucleo Reindex come lo fa il master. Il problema è che credo che lo schiavo pensi sia più recente, allora il master e il risultato è la replica è morto sul nucleo Reindex.

    Ecco alcuni messaggi di registro esattamente dopo il rilascio di uno scambio: 

    13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Master's version: 1328526603592, generation: 332
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Slave's version: 1328526603596, generation: 336
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Starting replication process
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Master's version: 1328526603596, generation: 336
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Slave's version: 1328526603592, generation: 332
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Starting replication process
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Number of files in latest index in master: 1
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Number of files in latest index in master: 11
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.tvx
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.fdt
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.frq
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/segments_9c
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.fdx
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.prx
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.tii
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.tvd
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.tis
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller$FileFetcher fetchPackets
WARNING: No content recieved for file: {name=segments_98, lastmodified=0, size=0}
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.tvf
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.fnm
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Total time taken for download : 0 secs
13-mrt-2012 14:29:46 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Total time taken for download : 0 secs
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller modifyIndexProps
INFO: New index installed. Updating index properties...
13-mrt-2012 14:29:46 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@598d00 main
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@598d00 main from Searcher@189c12a main
    fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@598d00 main
    fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@598d00 main from Searcher@189c12a main
    filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=34,warmupTime=1,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
13-mrt-2012 14:29:46 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
13-mrt-2012 14:29:46 org.apache.solr.handler.ReplicationHandler doFetch
SEVERE: SnapPull failed 
org.apache.solr.common.SolrException: Index fetch failed : 
    at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
    at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
    at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.RuntimeException: java.io.IOException: read past EOF
    at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1069)
    at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:418)
    at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:467)
    at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
    ... 10 more
Caused by: java.io.IOException: read past EOF
    at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:151)
    at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
    at org.apache.lucene.store.ChecksumIndexInput.readByte(ChecksumIndexInput.java:36)
    at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:68)
    at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:243)
    at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:76)
    at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
    at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:73)
    at org.apache.lucene.index.IndexReader.open(IndexReader.java:476)
    at org.apache.lucene.index.IndexReader.open(IndexReader.java:403)
    at org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:38)
    at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1058)
    ... 13 more
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@598d00 main
    filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=34,warmupTime=10,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
13-mrt-2012 14:29:46 org.apache.solr.core.SolrCore registerSearcher
INFO: [live] Registered new searcher Searcher@598d00 main
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher close
INFO: Closing Searcher@189c12a main
    fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
    filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=34,warmupTime=1,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller doCommit
INFO: Force open index writer to make sure older index files get deleted
13-mrt-2012 14:29:46 org.apache.solr.handler.ReplicationHandler doFetch
SEVERE: SnapPull failed 
org.apache.solr.common.SolrException: Index fetch failed : 
    at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
    at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
    at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:679)
Caused by: java.io.IOException: read past EOF
    at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:151)
    at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
    at org.apache.lucene.store.ChecksumIndexInput.readByte(ChecksumIndexInput.java:36)
    at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:68)
    at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:243)
    at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:309)
    at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
    at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:517)
    at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:306)
    at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1596)
    at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1418)
    at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:191)
    at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:98)
    at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:173)
    at org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpdateHandler2.java:376)
    at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:471)
    at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
    ... 10 more
    .

    Fino al punto 6, tutto va bene se faccio tutto manualmente. Tutto crolla dopo lo swap del nucleo.

    Spero che qualcuno potesse aiutarmi o dirmi cosa sto facendo male .. La replica dovrebbe essere disabilitata quando fai uno scambio di core? Qualche suggerimento?

È stato utile?

Soluzione

Sono riuscito a risolvere questo problema, il problema era che entrambi i miei core stavano usando lo stesso datadir.Impostando il datadir nel file solr.xml in entrambe le directory diverse ha risolto il problema!

Autorizzato sotto: CC-BY-SA insieme a attribuzione
Non affiliato a StackOverflow
scroll top