How To Backup AWS Elasticache Redis Data?

Question

I'm testing a new Redis cluster (AWS Large primary/slave nodes) in AWS Elasticache Redis.

How can I backup or export my data from the Redis cluster?

Other hosted redis services automatically create a Redis RDB dump file and store it on S3. I would like to do something similar with ElastiCache.

Answer 1

EDIT: New AWS Feature as of 4/24/2014

Amazon has added internal backup support as of 4/24. This allows you to snapshot redis data daily and restore it to an ElastiCache cluster. It does not allow exporting/downloading at the present. The solution below is still required if you want to keep your own archives/backups of redis data.

Most people should be able to use the built in backup system now available.

Original Answer

It looks like the only way to do this is to do the following.

• Spin up new EC2 instance
• Install Redis
• Configure local replica attached to ElastiCache redis primary
• Wait for data to sync locally
• Issue a redis SAVE command to generate a local dump
• Archive local dump.rdb on S3

I'd love a simpler solution or something built into ElastiCache.

EDIT: Elaboration

I ended up actually building this using the Ruby gem/utility "redis-backup" (https://github.com/josegonzalez/ruby-redis-backup) with a crontab running the following shell command:

sudo -u redis /usr/bin/env S3_SAVE=true redis-backup -s /var/lib/redis/dump.rdb -B YOUR_S3_BUCKET_FOR_BACKUPS -A S3_ACCESS_KEY_ID -S S3_ACCESS_SECRET_KEY

Answer 2

It doesn't look like this is possible anymore. According to the AWS docs,

"Beginning with Redis 2.8.22, ElastiCache no longer supports external read replicas."

Answer 3

ElastiCache may disable commands like save or bgsave. For reference, check Restricted Redis Commands.

The following bash script solution supports datatypes string and hash. If you want support for other data types (e.g., set, list, sorted set, bitmap), you have to extend the script (as commented in the script).

#!/bin/bash

# change KEY_PATTERN accordingly if you want a subset of the keys in redis cache
KEY_PATTERN="*"

# provide the default redis url here if you don't supply that from command line/environment:
if [[ -z $REDIS_SERVICE_URL ]]; then
    REDIS_SERVICE_URL=redis://localhost:6379
fi

IFS=' '

iKey=0 # counter to iterate over the keys
echo {

# iterate through the keys in redis:
while read key; do
    if [ $iKey -gt 0 ]; then
        echo ","
    fi

    echo \"$key\":

    # get the datatype for the current key
    datatype=$(redis-cli -u "$REDIS_SERVICE_URL" type "$key")

    # this script supports string/hash datatype.
    # Extend if you want to support other data types.
    if [ "$datatype" = 'string' ]; then
        val=$(redis-cli -u "$REDIS_SERVICE_URL" get "$key")
        echo -n \"${val//\"/\\\"}\"
    elif [ "$datatype" = 'hash' ]; then
        echo [
        i=0
        while read val; do
            if [ $i -gt 0 ]; then
                echo ","
            fi

            echo \"${val//\"/\\\"}\"

            i=$((i+1))
        done <<< $(redis-cli -u "$REDIS_SERVICE_URL" hgetall "$key")
        echo ]
    else
        echo Unsupported type $datatype
        exit -1
    fi

    iKey=$((iKey+1))
done <<< $(redis-cli -u "$REDIS_SERVICE_URL" keys "$KEY_PATTERN")

echo }

unset IFS

Formatted JSON of a sample output using above script:

{
    "vlaue-key-1": "value-1",
    "hash-key-1": [
        "key-a",
        "value of key-a in hash-key-1",
        "key-b",
        "20"
    ],
    "vlaue-key-2": "25.5",
    "hash-key-2": [
        "key-x",
        "value-x of hash-key-2/key-x",
        "key-b",
        "9999"
    ]
}