Solr 4.1 DataImportHandler ClassNotFoundException

StackOverflow https://stackoverflow.com/questions/14827375

  •  09-03-2022
  •  | 
  •  

Domanda

I have been trying to setup Data Import Handler (Solr 4.1) following this tutorial and tried solutions suggested in previous posts such as Configure DIH in multicore solr and added the dataimport jar to the classpath but the error still persists. Any methods to solve this?

Here is the entire exception stacktrace:

SEVERE: Unable to create core: collection1
org.apache.solr.common.SolrException: RequestHandler init failure
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:607)
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:1003)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1033)
at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629)
at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:624)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.apache.solr.common.SolrException: RequestHandler init failure
at org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:168)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:731)
... 13 more
Caused by: org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.handler.dataimport.DataImportHandler'
at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:438)
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:507)
at org.apache.solr.core.SolrCore.createRequestHandler(SolrCore.java:581)
at    org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:154)
... 14 more
 Caused by: java.lang.ClassNotFoundException: org.apache.solr.handler.dataimport.DataImportHandler
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:627)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:422)
... 17 more
Feb 12, 2013 12:37:47 PM org.apache.solr.common.SolrException log
SEVERE: null:org.apache.solr.common.SolrException: Unable to create core: collection1
at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1654)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1039)
at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629)
at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:624)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.apache.solr.common.SolrException: RequestHandler init failure
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:607)
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:1003)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1033)
... 10 more
Caused by: org.apache.solr.common.SolrException: RequestHandler init failure
at      org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:168)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:731)
... 13 more
Caused by: org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.handler.dataimport.DataImportHandler'
at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:438)
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:507)
at org.apache.solr.core.SolrCore.createRequestHandler(SolrCore.java:581)
at org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:154)
... 14 more
Caused by: java.lang.ClassNotFoundException: org.apache.solr.handler.dataimport.DataImportHandler
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:627)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:422)
... 17 more

Solrconfig.xml :

    <?xml version="1.0" encoding="UTF-8" ?>
    <config>
    <luceneMatchVersion>LUCENE_41</luceneMatchVersion>
    <lib dir="../../../contrib/extraction/lib" regex=".*\.jar" />
    <lib dir="../../../dist/" regex="solr-cell-\d.*\.jar" />
    <lib dir="../../../contrib/clustering/lib/" regex=".*\.jar" />
    <lib dir="../../../dist/" regex="solr-clustering-\d.*\.jar" />
    <lib dir="../../../contrib/langid/lib/" regex=".*\.jar" />
    <lib dir="../../../dist/" regex="solr-langid-\d.*\.jar" />
    <lib dir="../../../contrib/velocity/lib" regex=".*\.jar" />
    <lib dir="../../../dist/" regex="solr-velocity-\d.*\.jar" />
    <lib dir="../../../dist/" regex="solr-dataimporthandler-4.1.0.jar\d.*\.jar" />
    <lib dir="../../lib/" regex=".*\.jar" /> 
    <dataDir>${solr.data.dir:}</dataDir>
    <directoryFactory name="DirectoryFactory" 
        class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/> 
    <indexConfig>
    </indexConfig>
    <jmx />
    <updateHandler class="solr.DirectUpdateHandler2">
        <updateLog>
            <str name="dir">${solr.ulog.dir:}</str>
        </updateLog>
        <autoCommit> 
            <maxTime>15000</maxTime> 
            <openSearcher>false</openSearcher> 
        </autoCommit>
    </updateHandler>
    <query>
        <maxBooleanClauses>1024</maxBooleanClauses>
        <filterCache class="solr.FastLRUCache"
            size="512"
            initialSize="512"
            autowarmCount="0"/>
        <queryResultCache class="solr.LRUCache"
            size="512"
            initialSize="512"
            autowarmCount="0"/>
        <documentCache class="solr.LRUCache"
            size="512"
            initialSize="512"
            autowarmCount="0"/>
        <enableLazyFieldLoading>true</enableLazyFieldLoading>
        <queryResultWindowSize>20</queryResultWindowSize>
        <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
        <listener event="newSearcher" class="solr.QuerySenderListener">
            <arr name="queries">
            </arr>
        </listener>
        <listener event="firstSearcher" class="solr.QuerySenderListener">
            <arr name="queries">
                <lst>
                    <str name="q">static firstSearcher warming in solrconfig.xml</str>
                </lst>
            </arr>
        </listener>
        <useColdSearcher>false</useColdSearcher>
        <maxWarmingSearchers>2</maxWarmingSearchers>
    </query>
    <requestDispatcher handleSelect="false" >
        <requestParsers enableRemoteStreaming="true" 
            multipartUploadLimitInKB="2048000"
            formdataUploadLimitInKB="2048"/>
        <httpCaching never304="true" />
    </requestDispatcher>
    <requestHandler name="/select" class="solr.SearchHandler">
        <lst name="defaults">
            <str name="echoParams">explicit</str>
            <int name="rows">10</int>
            <str name="df">app_data</str>
        </lst>
    </requestHandler>
    <requestHandler name="/query" class="solr.SearchHandler">
        <lst name="defaults">
            <str name="echoParams">all</str>
            <str name="wt">xml</str>
            <str name="indent">true</str>
            <str name="df">app_data</str>
            <str name="spellcheck.dictionary">default</str>
            <str name="spellcheck.dictionary">wordbreak</str>
            <str name="spellcheck">on</str>
            <str name="spellcheck.extendedResults">true</str>       
            <str name="spellcheck.count">10</str>
            <str name="spellcheck.alternativeTermCount">5</str>
            <str name="spellcheck.maxResultsForSuggest">5</str>       
            <str name="spellcheck.collate">true</str>
            <str name="spellcheck.collateExtendedResults">true</str>  
            <str name="spellcheck.maxCollationTries">10</str>
            <str name="spellcheck.maxCollations">5</str>         
        </lst>
        <arr name="last-components">
            <str>spellcheck</str>
        </arr>
    </requestHandler>
    <requestHandler name="/get" class="solr.RealTimeGetHandler">
        <lst name="defaults">
            <str name="omitHeader">true</str>
            <str name="wt">json</str>
            <str name="indent">true</str>
        </lst>
    </requestHandler>
    <requestHandler name="/browse" class="solr.SearchHandler">
    </requestHandler>
    <requestHandler name="/update" class="solr.UpdateRequestHandler">
    </requestHandler>
    <requestHandler name="/update/json" class="solr.JsonUpdateRequestHandler">
        <lst name="defaults">
            <str name="stream.contentType">application/json</str>
        </lst>
    </requestHandler>
    <requestHandler name="/update/csv" class="solr.CSVRequestHandler">
        <lst name="defaults">
            <str name="stream.contentType">application/csv</str>
        </lst>
    </requestHandler>
    <requestHandler name="/update/extract" 
        startup="lazy"
        class="solr.extraction.ExtractingRequestHandler" >
        <lst name="defaults">
            <str name="lowernames">true</str>
            <str name="uprefix">ignored_</str>
            <str name="captureAttr">true</str>
            <str name="fmap.a">links</str>
            <str name="fmap.div">ignored_</str>
        </lst>
    </requestHandler>
    <requestHandler name="/analysis/field" 
        startup="lazy"
        class="solr.FieldAnalysisRequestHandler" />
    <requestHandler name="/analysis/document" 
        class="solr.DocumentAnalysisRequestHandler" 
        startup="lazy" />
    <requestHandler name="/admin/" 
        class="solr.admin.AdminHandlers" />
    <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
        <lst name="invariants">
            <str name="q">solrpingquery</str>
        </lst>
        <lst name="defaults">
            <str name="echoParams">all</str>
        </lst>
    </requestHandler>
    <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
        <lst name="defaults">
            <str name="echoParams">explicit</str> 
            <str name="echoHandler">true</str>
        </lst>
    </requestHandler>
    <requestHandler name="/replication" class="solr.ReplicationHandler" > 
    </requestHandler>
    <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
        <str name="queryAnalyzerFieldType">text_general</str>
        <lst name="spellchecker">
            <str name="name">default</str>
            <str name="field">keyword</str>
            <str name="classname">solr.DirectSolrSpellChecker</str>
            <str name="distanceMeasure">internal</str>
            <float name="accuracy">0.5</float>
            <int name="maxEdits">2</int>
            <int name="minPrefix">0</int>
            <int name="maxInspections">10</int>
            <int name="minQueryLength">4</int>
            <float name="maxQueryFrequency">0.02</float><!--appears in more than 2% of the documents its assumed to be correct, and no suggestions are given, I don't think use of this parameter makes any sense since dictionary is based on keyword field which is unique. Not sure if frequency is also based on that column only.-->
        </lst>
        <lst name="spellchecker">
            <str name="name">wordbreak</str>
            <str name="classname">solr.WordBreakSolrSpellChecker</str>      
            <str name="field">keyword</str>
            <str name="combineWords">true</str>
            <str name="breakWords">true</str>
            <int name="maxChanges">10</int>
        </lst>
    </searchComponent>
    <requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
        <lst name="defaults">
            <str name="df">app_data</str>
            <str name="spellcheck.dictionary">default</str>
            <str name="spellcheck.dictionary">wordbreak</str>
            <str name="spellcheck">on</str>
            <str name="spellcheck.extendedResults">true</str>       
            <str name="spellcheck.count">10</str>
            <str name="spellcheck.alternativeTermCount">5</str>
            <str name="spellcheck.maxResultsForSuggest">5</str>       
            <str name="spellcheck.collate">true</str>
            <str name="spellcheck.collateExtendedResults">true</str>  
            <str name="spellcheck.maxCollationTries">10</str>
            <str name="spellcheck.maxCollations">5</str>         
        </lst>
        <arr name="last-components">
            <str>spellcheck</str>
        </arr>
    </requestHandler>
    <searchComponent name="tvComponent" class="solr.TermVectorComponent"/>
    <requestHandler name="/tvrh" class="solr.SearchHandler" startup="lazy">
        <lst name="defaults">
            <str name="df">app_data</str>
            <bool name="tv">true</bool>
        </lst>
        <arr name="last-components">
            <str>tvComponent</str>
        </arr>
    </requestHandler>
    <searchComponent name="clustering"
        enable="${solr.clustering.enabled:false}"
        class="solr.clustering.ClusteringComponent" >
        <lst name="engine">
            <str name="name">default</str>
            <str name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
            <str name="LingoClusteringAlgorithm.desiredClusterCountBase">20</str>
            <str name="carrot.lexicalResourcesDir">clustering/carrot2</str>
            <str name="MultilingualClustering.defaultLanguage">ENGLISH</str>
        </lst>
        <lst name="engine">
            <str name="name">stc</str>
            <str name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm</str>
        </lst>
    </searchComponent>
    <requestHandler name="/clustering"
        startup="lazy"
        enable="${solr.clustering.enabled:false}"
        class="solr.SearchHandler">
        <lst name="defaults">
            <bool name="clustering">true</bool>
            <str name="clustering.engine">default</str>
            <bool name="clustering.results">true</bool>
            <str name="carrot.title">name</str>
            <str name="carrot.url">id</str>
            <str name="carrot.snippet">features</str>
            <bool name="carrot.produceSummary">true</bool>
            <bool name="carrot.outputSubClusters">false</bool>
            <str name="defType">edismax</str>
            <str name="qf">
                app_data
            </str>
            <str name="q.alt">*:*</str>
            <str name="rows">10</str>
            <str name="fl">*</str>
        </lst>     
        <arr name="last-components">
            <str>clustering</str>
        </arr>
    </requestHandler>
    <searchComponent name="terms" class="solr.TermsComponent"/>
    <requestHandler name="/terms" class="solr.SearchHandler" startup="lazy">
        <lst name="defaults">
            <bool name="terms">true</bool>
            <bool name="distrib">false</bool>
        </lst>     
        <arr name="components">
            <str>terms</str>
        </arr>
    </requestHandler>
    <searchComponent name="elevator" class="solr.QueryElevationComponent" >
        <str name="queryFieldType">string</str>
        <str name="config-file">elevate.xml</str>
    </searchComponent>
    <requestHandler name="/elevate" class="solr.SearchHandler" startup="lazy">
        <lst name="defaults">
            <str name="echoParams">explicit</str>
            <str name="df">app_data</str>
        </lst>
        <arr name="last-components">
            <str>elevator</str>
        </arr>
    </requestHandler>
      <requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
    <lst name="defaults">
      <str name="config">/Users/username/data-config.xml</str>
    </lst>
  </requestHandler>
    <searchComponent class="solr.HighlightComponent" name="highlight">
        <highlighting>
            <fragmenter name="gap" 
                default="true"
                class="solr.highlight.GapFragmenter">
                <lst name="defaults">
                    <int name="hl.fragsize">100</int>
                </lst>
            </fragmenter>
            <fragmenter name="regex" 
                class="solr.highlight.RegexFragmenter">
                <lst name="defaults">
                    <int name="hl.fragsize">70</int>
                    <float name="hl.regex.slop">0.5</float>
                    <str name="hl.regex.pattern">[-\w ,/\n\&quot;&apos;]{20,200}</str>
                </lst>
            </fragmenter>
            <formatter name="html" 
                default="true"
                class="solr.highlight.HtmlFormatter">
                <lst name="defaults">
                    <str name="hl.simple.pre"><![CDATA[<em>]]></str>
                        <str name="hl.simple.post"><![CDATA[</em>]]></str>
                </lst>
            </formatter>
            <encoder name="html" 
                class="solr.highlight.HtmlEncoder" />
            <fragListBuilder name="simple" 
                class="solr.highlight.SimpleFragListBuilder"/>
            <fragListBuilder name="single" 
                class="solr.highlight.SingleFragListBuilder"/>
            <fragListBuilder name="weighted" 
                default="true"
                class="solr.highlight.WeightedFragListBuilder"/>
            <fragmentsBuilder name="default" 
                default="true"
                class="solr.highlight.ScoreOrderFragmentsBuilder">
            </fragmentsBuilder>
            <fragmentsBuilder name="colored" 
                class="solr.highlight.ScoreOrderFragmentsBuilder">
                <lst name="defaults">
                    <str name="hl.tag.pre"><![CDATA[
                        <b style="background:yellow">,<b style="background:lawgreen">,
                                <b style="background:aquamarine">,<b style="background:magenta">,
                                        <b style="background:palegreen">,<b style="background:coral">,
                                                <b style="background:wheat">,<b style="background:khaki">,
                                                        <b style="background:lime">,<b style="background:deepskyblue">]]></str>
                                                            <str name="hl.tag.post"><![CDATA[</b>]]></str>
                                                    </lst>
                                                </fragmentsBuilder>
                                                <boundaryScanner name="default" 
                                                    default="true"
                                                    class="solr.highlight.SimpleBoundaryScanner">
                                                    <lst name="defaults">
                                                        <str name="hl.bs.maxScan">10</str>
                                                        <str name="hl.bs.chars">.,!? &#9;&#10;&#13;</str>
                                                    </lst>
                                                </boundaryScanner>
                                                <boundaryScanner name="breakIterator" 
                                                    class="solr.highlight.BreakIteratorBoundaryScanner">
                                                    <lst name="defaults">
                                                        <str name="hl.bs.type">WORD</str>
                                                        <str name="hl.bs.language">en</str>
                                                        <str name="hl.bs.country">US</str>
                                                    </lst>
                                                </boundaryScanner>
                                            </highlighting>
                                        </searchComponent>
                                        <queryResponseWriter name="json" class="solr.JSONResponseWriter">
                                            <str name="content-type">text/plain; charset=UTF-8</str>
                                        </queryResponseWriter>
                                        <queryResponseWriter name="velocity" class="solr.VelocityResponseWriter" startup="lazy"/>
                                        <queryResponseWriter name="xslt" class="solr.XSLTResponseWriter">
                                            <int name="xsltCacheLifetimeSeconds">5</int>
                                        </queryResponseWriter>
                                        <admin>
                                            <defaultQuery>*:*</defaultQuery>
                                        </admin>
                                    </config>
È stato utile?

Soluzione

The best way to handle any class not found issue is to grab the required jars and put them in solr/lib dir, where the solr dir is same that contains solr.xml used to configure solr cores you will also need to add following line to solrconfig.xml

<lib dir="../lib" />

This will load all required libs for all cores on your solr installation.

If you need to restrict the jar to a core then put it int

instanceDir/lib , no configuration is required for solrconfig.xml in this case.

for data import handler to work, you will need the JDBC jar also as well as dist/solr-dataimporthandler-*.jar to be put into the lib dir that you configure

Altri suggerimenti

Add lib folder in your Solr Core directory and add the required lib jar file into this. In this case you do not need to specify lib path in solrconfig.xml file.

In solr 4.1.0 add the following jar files in the project.
solr-dataimporthandler-4.1.0, solr-dataimporthandler-extras-4.1.0

Execute it again.It will work.

I just started trying to get Solr 4.1 working with MySQL today and encountered the exact same problem as you.

I managed to solve it by doing the following:

  1. Commented out all the default LIB directories except for one:

    <lib dir="../../../contrib/extraction/lib" regex=".*\.jar" /> <!-- <lib dir="../../../dist/" regex="solr-cell-\d.*\.jar" /> <lib dir="../../../contrib/clustering/lib/" regex=".*\.jar" /> <lib dir="../../../dist/" regex="solr-clustering-\d.*\.jar" /> <lib dir="../../../contrib/langid/lib/" regex=".*\.jar" /> <lib dir="../../../dist/" regex="solr-langid-\d.*\.jar" /> <lib dir="../../../contrib/velocity/lib" regex=".*\.jar" /> <lib dir="../../../dist/" regex="solr-velocity-\d.*\.jar" /> -->

  2. Grabbed the latest mysql connector jarr file and solr dataimporthandler jar

  3. Placed them in the remaining directory.

Started the server again and it worked.

This is on Windows 8 X86_64.

Hope this helps you.

The selected answer certainly works (which pretty much boils down to "copy everything to $instancedir/lib") but solr configuration provides a way to separate the "standard/global" libraries that comes with solr and "extension" libraries (i.e., custom solr handlers/parsers/etc) that's specific to the instance.

there are at least two directories that solr looks into to load classes needs to load: the ${solr.install.dir} which is pretty much where solr is installed. use the scripts (available in v4.10.x) this is set by default to the directory where bin/solr script is run. solrconfig.xml uses this directory as the base for loading the jars specified in the solrconfig.xml file; it can be overriden by a java -D parameter or overriding completely in the file itself as specified in the selected answer. this is i would think one would have to load the "global" libraries.

now, there's a 2nd solr variable ${solr.solr.home} which is the instance home where all the index artifacts/files are created, and here we have solr.xml. you have this directory for each solr instance (running in a different port) separately. in that solr.xml file, you can have a sharedLib element, which is a folder relative ${solr.solr.home}. this is i would think any "extension" libraries specific to a solr instance will be loaded from.

I had the same exception .add solr-dataimporthandler-extras-4.7.2.jar and solr-dataimporthandler-4.7.2.jar in your core's lib folder. This worked for me .

Autorizzato sotto: CC-BY-SA insieme a attribuzione
Non affiliato a StackOverflow
scroll top