ServiceStack.Redis SearchKeys

Question

I am using the ServiceStack.Redis client on C#.

I added about 5 million records of type1 using the following pattern a::name::1 and 11 million records of type2 using the pattern b::RecId::1.

Now I am using redis typed client as client = redis.As<String>. I want to retrieve all the keys of type2. I am using the following pattern:

var keys = client.SearchKeys("b::RecID::*");

But it takes forever (approximately 3-5 mins) to retrieve the keys.

Is there any faster and more efficient way to do this?

Answer 1

You should work hard to avoid the need to scan the keyspace. KYES is literally a server stopper, but even if you have SCAN available: don't do that. Now, you could choose to keep the keys of things you have available in a set somewhere, but there is no SRANGE etc - in 2. you'd have to use SMEMBERS, which is still going to need to return a few million records - but at least they will all be available. In later server versions, you have access to SCAN (think: KEYS) and SSCAN (think: SMEMBERS), but ultimately you simply have the issue of wanting millions of rows, which is never free.

If possible, you could mitigate the impact by using a master/slave pair, and running the expensive operations on the slave. At least other clients will be able to do something while you're killing the server.

Answer 2

The keys command in Redis is slow (well, not slow, but time consuming). It also blocks your server from accepting any other command while it's running.

If you really want to iterate over all of your keys take a look at the scan command instead- although I have no idea about ServiceStack for this

Answer 3

You can use the SCAN command, make a loop search, where each search is restricted to a smaller number of keys. For a complete example, refer to this article: http://blog.bossma.cn/csharp/nservicekit-redis-support-scan-solution/