Redis hash very slow writing speed

Question

I'm facing a very strange issue : i get really crappy writing speeds when using redis (in a ideal world the writing speed should be approaching the writing speed on RAM).

Here is my benchmark :

package redisbenchmark;
import redis.clients.jedis.Jedis;

public class RedisBenchmark {

    private static final String REDIS_KEY = "anon_id";

    private Jedis conn;

    private long writeTimeNano=0;

    private RandomString stringGenerator;

    private String[] fields;

    public RedisBenchmark(){
        conn = new Jedis("localhost");
        stringGenerator = new RandomString(32);
    }

    public void run(int nbWrites, int nbReads){     
        writeBenchmark(nbWrites);
    }

    public void writeBenchmark(int amount){
        fields = new String[amount];

        for(int i=0; i< amount; i++){
            fields[i] = stringGenerator.nextString();           
        }

        long start = System.nanoTime();
        for(int i=0; i< amount; i++){
            write(fields[i]);
        }
        writeTimeNano+=System.nanoTime()-start;

        double seconds = (double)writeTimeNano / 1000000000.0;
        System.out.println("[write]nb:"+amount+"|time:"+seconds+"|speed:"+((amount*33)/(seconds*1024*1024))+" MB/s");
    }

    public void write(String anonId){       
        conn.hsetnx(REDIS_KEY, anonId, "1");
    }


    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        // TODO code application logic here
        RedisBenchmark benchmark = new RedisBenchmark();
        benchmark.run(100000, 200);
    }
}

RandomString is a class that generates a random string (the arg is the string length)

And below are a couple of results :

[write]nb:100000|time:4.408319378|speed:0.713905907055318 MB/s [write]nb:100000|time:4.447246995|speed:0.707656949946542 MB/s

I tried to modify the save to hdd parameters in the config file but with no improvement.

I have 2 ideas:
1. Its a socket problem since client and server (redis) are on the same machine
2. The connector implementation has performance issues

UPDATE The benchmark results for set operation:

====== SET ======
10000 requests completed in 0.09 seconds
50 parallel clients
3 bytes payload
keep alive: 1

99.51% <= 1 milliseconds
100.00% <= 1 milliseconds
111111.11 requests per second

System specification :
- Ubuntu 11.04
- 8GB RAM
- Intel i5 processor

Any suggestion will be greatly appreciated.

Answer 1

You need to think a bit more about what you really benchmark using this program. I can tell you it is not Redis, but rather the capability of your system to run a ping pong game between two processes (because all your hsetnx calls are synchronous).

Please read this page before trying to benchmark Redis, it will definitely help you.

Your assumption that the speed of Redis should approach the writing speed of RAM is somewhat naive. Redis is a remote store, and for O(1) operations, most of the overhead is due to the communication costs. For synchronous traffic (like your example), it is also due to the cost of the OS scheduler.

If you want to apply of lot of commands in sequence, you need to use pipelining. Or if you do not care about the sequence, you can work concurrently with several connections (this is the default mode for redis-benchmark). Or you can try to send asynchronous commands instead. In all cases, the idea is to amortize the cost of the roundtrips to the Redis server

With pipelining on several connections with asynchronous traffic, you will get the maximum throughput Redis can achieve on this machine.