Java Spring jdbcTemplate queryForList slow

Question

I'm using Spring jdbcTemplate to run MySQL queries in my java application. This is my function:

public static ArrayList<Map<String, Object>> query(String q) throws Exception {

    JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);

    @SuppressWarnings("unchecked")
    List<Map<String, Object>> result = jdbcTemplate.queryForList(q);

    return (ArrayList<Map<String, Object>>) result;

}

In this particular case, I pass the function this query:

"SELECT * FROM test WHERE name IN('string1', 'string2', 'string3', ...)";

Table test only has 6 columns, strings in IN vary from few to 100 symbols. jdbcTemplate.queryForList() takes 280 miliseconds to run the query. NOTE: it takes 280 miliseconds for jdbcTemplate.queryForList() to run, not the whole function query. When I run exactly the same query on MySQL client (HeidiSQL) it only takes 16 miliseconds. UPDATE: Heidi maintains open connection, so that is not a fair comparison.

Question: why is jdbcTemplate.queryForList() so miserably slow in comparison to the same query ran on MySQL client? I'm using it purely for convienence reasons, it can retrieve result in ArrayList<Map<String, Object>> which is what I need. Is there anything I can do to speed it up or should I just drop jdbcTemplate all together and use something else?

UPDATE: OK, so I've changed my function from jdbcTemplate to using plain jdbc. This is how that abomination looks like. It's the reason why I went with jdbcTemplate in the first place. It now takes 200 miliseconds to run the same query - slight improvement, but still too slow...

public static ArrayList<Map<String,Object>> query(String Full_Command) {

    try {

        String URL = "jdbc:mysql://localhost/dbname";
        String USER = "root";
        String PASS = "";
        java.sql.Connection Con = DriverManager.getConnection(URL, USER, PASS);

        //create statement
        Statement Stm = null;
        //Stm = Con.createStatement();
        Stm = (Statement) Con.createStatement();

        //query
        ResultSet Result = null;
        boolean Returning_Rows = Stm.execute(Full_Command);
        if (Returning_Rows) {
            Result = Stm.getResultSet();
        } else {
            return new ArrayList<Map<String,Object>>();
        }

        //get metadata
        ResultSetMetaData Meta = null;
        Meta = Result.getMetaData();

        //get column names
        int Col_Count = Meta.getColumnCount();
        ArrayList<String> Cols = new ArrayList<String>();
        for (int Index=1; Index<=Col_Count; Index++) {
            Cols.add(Meta.getColumnName(Index));
        }

        //fetch out rows
        ArrayList<Map<String, Object>> Rows = new ArrayList<Map<String,Object>>();

        while (Result.next()) {
            HashMap<String,Object> Row = new HashMap<String,Object>();
            for (String Col_Name:Cols) {
                Object Val = Result.getObject(Col_Name);
                Row.put(Col_Name,Val);
            }
            Rows.add(Row);
        }

        //close statement
        Stm.close();

        //pass back rows
        return Rows;

    } catch (Exception Ex) {

        System.out.print(Ex.getMessage());
        return new ArrayList<Map<String,Object>>();

    }

}

UPDATE2: I have not broken down JDBC function into execution times, and this single line is the bottleneck, taking ~190 miliseconds every time:

java.sql.Connection Con = DriverManager.getConnection(URL, USER, PASS);

Any comments?

Answer 1

It's not JdbcTemplate's fault. Here's why:

First of all, you have to discount the overhead in establishing a connection. This overhead can be as much as 500 milliseconds. This is normal.

Second of all, when you run the query on MySQL client, are you running it immediately after you ran your program? Because queries get cached... even at the level of disk I/O. So if you run the same query twice, the second time it will run faster due to caching.

Third of all, are you indexing the column name?

Fourth of all, if the connection overhead really matters that much to you, you could consider conserving a connection or even connection pooling.

Answer 2

Try to increase fetch size (JdbcTemplate.setFetchSize), it might be the bottleneck.

Answer 3

I dont think you can get any faster than raw jdbc call. When you say that you are able fetch it faster with sql client then is it in first attempt? Client and db can cache results, so check then timing of first query run from your sql client after session is created. Also post java logs , that will help to see where time is spent.

Answer 4

Could be the overhead to establish your first DB connection. Try benchmarking performance after you've definitely already got a connection.

This stuff shouldn't be slow, the gains are in there somewhere!