Storing an Event Timeline in Redis

Question

I'd like to store a list of events in a timeline in Redis. I'm thinking of a sorted set with seconds since 1970 as the score, allowing quick lookup and range searches.

The problem is I want each of these events to represent an object, with a couple of keys (about 3 maybe), so I'd store a sorted set of IDs - the application would then lookup the hash based on the ID.

So the conversation would go something like this:

> ZRANGEBYSCORE events start end 
1) "16"
2) "17"
> HGETALL events:16
1) "key1"
2) "val1"
...
> HGETALL events:17
...

Is there a better way to organise or do things that would avoid the application making a separate call for each result. Could everything be done on the redis-server side to avoid this overhead? Is it even a big problem?

PS. I'm using the node client, but my question applies to other languages too.

Answer 1

An obvious solution is to work by value instead of working by reference. So your zset could store serialized objects instead of storing the references on these objects. With node, JSON is easy to use.

Now, if you need to keep the current structure (because for instance some of your objects are shared by various zsets), then you need to think in term of rountrips rather than commands. What is expensive in Redis is the roundtrip, not really the execution of the O(1) command itself.

HGETALL does not support variadic parameters (i.e. you can only retrieve one hash object). But Redis is quite good at processing pipelined commands, and this is especially easy to do from the node_redis driver due to the asynchronous nature of node. So in your example, you just need 2 roundtrips: the first one to perform the ZRANGEBYSCORE, and a second one to perform all the HGETALL commands (i.e. do not wait the result of the first HGETALL command before issuing the second one). It will be quite efficient.

With the current Redis version (2.4), there is no way to eliminate the second roundtrip.

Answer 2

You could stringify the object and store it as a stringwith JSON.stringify() and then retrieve it with JSON.parse()