Performance issue between solr 1.4.0 and 4.6.0

Question

I updated my solr version from 1.4.0 to 4.6.0 and now we are facing several performance issues.

a) If I use embedded version, it's very slow

b) Using http, I have these average times:

• 1.4: 151ms
• 4.6: 301ms

c) I saw that JavaBinCodec changed from version 1 to 2. Anybody nows if this can be the problem?

Note1: I tested many times, discarding first time, because of the warm up of server.

Note2: The documents returned are very big (3k lines in XML view, each document)

Any help would be apreciated.

---

The code used to test, showing code to solr 4.6

public class Main {

    private static HttpSolrServer server;
    public static void main(String[] args) throws Exception {
        String url = "http://foo.bar/myIndex";
        server = new HttpSolrServer(url);
        for (int i = 0; i < 10; i++) {
            search();
        }
    }
    public static void search() throws Exception {
        SolrQuery solrQuery = new SolrQuery();
        solrQuery.setQuery("foo:bar");
        solrQuery.setStart(0);
        solrQuery.setRows(20);

        // QUERY
        long before = new GregorianCalendar().getTimeInMillis();
        server.query(solrQuery);
        long after = new GregorianCalendar().getTimeInMillis();
        System.out.println(after - before);
    }

}

Answer 1

Solr 4.6 runs on Java 6 or higher version. When using Java 7, Solr recommends to install at least Update 1 and also discourages the experimental usage of -XX JVM options. Latest version of JVMs may affect the performance of Solr. You can have an overview of issues in Solr due to JVM at the link below.

http://wiki.apache.org/lucene-java/JavaBugs

CPU, disk and memory requirements are based on the many choices made in implementing Solr (document size, number of documents, and number of hits retrieved to name a few).

However, you can try several things to improve the performance of Solr if you are using Zookeeper.

1. Move Zookeeper, if using, to another disk. If the index is huge then number of I/O call from Solr to Zookeeper will degrade the assembly performance.
2. Increase the Zookeeper timeout period.
3. Log gc times, I found out pauses of upto 20s on Zookeeper boxes.
4. Use the recommendations to tune the heap from http://wiki.apache.org/solr/ShawnHeisey#GC_Tuning.