Redis - handling changes to data structures

Question

I have been experimenting with Redis, and I really like the scalability that it brings to the table. However, I'm wondering how to handle changes to data structures for a system that's already in production.

For example, let me say that I am collecting information about a user, and I use the user_id as a key, and dumping the other data about the user as comma separated values.

user_id: name, email, etc.

Now, say after about 100,000 records, I realise that I needed to query by email - how would I now take a snapshot of the existing data and create a new index for it?

Answer 1

From my understanding of Redis, this would require something which Redis is not designed to do. You would have to loop though all your records (using keys *) and then change the order of the data and make a new key. I, personally, would recommend using a list instead of a comma separated string. In a list, you can reorder it from inside redis. A Redis List looks like the following:

"Colum" => [0] c.mcgaley@gmail.com
            [1] password
            [2] Something

I am building an app in which I encountered the same problem. I solved it by having a list for all the user's info, and then have a key with the user's email with a value of the user's id. So my database would something like this:

"Colum" => [0] c.mcgaley@gmail.com
            [1] password
            [2] Something
"c.mcgaley@gmail.com" => "Colum"

So I could query the ID or the Email and still get the information I needed.

Sorry that I was not able to directly answer your question. Just hope this helped.

Answer 2

Using csv is not a great idea if you want to support changes. You need to use a serializer that handles missing/new values if everything is in one key, or you can use a redis hash, which gives you named subkeys. Either way you can add/remove fields with the only requirement being that your code knows what to do if it reads a record without the new value.

To allow lookup by email you need to add an index - basically a key (or list) for each email with the user id as the value. You will need to populate this index by getting all keys once, then making sure you update it when emails change.

You could iterate over all keys and store them with a different id, but that is probably more trouble than it is worth.