Elasticsearch "Client only" dependency in Maven

Question

We need to include a elasticsearch client into a project which itself uses Lucene for other indexing/storage tasks. Including the whole library results in a dependency conflict because the Lucene versions are not the same (ES uses 4.7, we use 4.0). Is there any "client only" package of elasticsearch or does anybody can think of any other solution?

Edit:

The approach to exclude all the lucene packages resulted in the following error:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/lucene/util/Version
    at org.elasticsearch.Version.<clinit>(Version.java:42)
    at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:169)
    at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:125)
    at de.tensquare.wpt.entrysearchindex.ElasticSearchClient.<init>(ElasticSearchClient.java:74)
    at de.tensquare.wpt.entrysearchindex.SearchIndex.<init>(SearchIndex.java:81)
    at de.tensquare.wpt.entrysearchindex.SearchIndex.main(SearchIndex.java:152)
Caused by: java.lang.ClassNotFoundException: org.apache.lucene.util.Version
    at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

Answer 1

I recently ran into similar issues. We have both solr and elasticsearch in our stack. Using elasticserch transport client, we were ran into lucence snafus. Here is the workaround that helped me. I created a shaded jar that changes how elasticsearch references lucene and carrot search packages.

EDIT: I ran into issue from another project, so needed to shade a bit further, updated solution

elasticsearch wraper project pom

<?xml version="1.0"?>
<project .>
    <parent>
        <artifactId>someartifactId</artifactId>
        <groupId>some.group.id</groupId>
        <version>1.0.0</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <artifactId>someartifactId-elasticsearch</artifactId>
    <name>Some Artifact Elasticsearch</name>
    <packaging>jar</packaging>
    <dependencies>
        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>2.1.2</version>
        </dependency>
    </dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-source-plugin</artifactId>
            <executions>
                <execution>
                    <id>attach-sources</id>
                    <goals>
                        <goal>jar</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>2.4</version>
            <configuration>
                <filters>
                    <filter>
                        <artifact>*:*</artifact>
                        <excludes>
                            <exclude>META-INF/services/org.apache.lucene.codecs.Codec</exclude>
                            <exclude>META-INF/services/org.apache.lucene.codecs.DocValuesFormat</exclude>
                            <exclude>META-INF/services/org.apache.lucene.codecs.PostingsFormat</exclude>
                            <exclude>META-INF/services/org.apache.lucene.analysis.util.CharFilterFactory</exclude>
                            <exclude>META-INF/services/org.apache.lucene.analysis.util.TokenFilterFactory</exclude>
                            <exclude>META-INF/services/org.apache.lucene.analysis.util.TokenizerFactory</exclude>
                        </excludes>
                    </filter>
                </filters>
            </configuration>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <relocations>
                            <relocation>
                                <pattern>org.</pattern>
                                <shadedPattern>hack.org.</shadedPattern>
                            </relocation>
                            <relocation>
                                <pattern>com.</pattern>
                                <shadedPattern>hack.com.</shadedPattern>
                            </relocation>
                        </relocations>
                        <transformers>
                            <transformer
                                implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
                        </transformers>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <artifactId>maven-assembly-plugin</artifactId>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>single</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <descriptorRefs>
                    <descriptorRef>jar-with-dependencies</descriptorRef>
                </descriptorRefs>

            </configuration>
        </plugin>
    </plugins>
</build>
</project>

In src/main/resources/META-INF/services, create shaded filters and codecs that get packaged up

hack.org.apache.lucene.analysis.util.CharFilterFactory
hack.org.apache.lucene.analysis.util.TokenFilterFactory
hack.org.apache.lucene.codecs.Codec
hack.org.apache.lucene.codecs.DocValuesFormat
hack.org.apache.lucene.codecs.PostingsFormat
hack.org.apache.lucene.analysis.util.TokenizerFactory

Now to use the TransportClient (or any other elastic search class) use package hack.org.elasticsearch....

Answer 2

Include elasticsearch.1.1.0.jar and lucune.4.7.jar in class path. its enough for transport Client.

HOpe it helps..!