Amazon SQS to store data

Question

I am developing a service that use Amazon SQS service to store data. I would like to save the data into key, value structure so client thats going to consume the queue should send the key to get the data. Its possible to maintain this kind of structure or suggest any other appropriate Amazon service for following requirement.

Answer 1

SQS will not work for several reasons:

• SQS has a maximum storage time of 14 days
• SQS messages have a maximum storage size of 256 kb
• When a client receives messages, they do not control the order they receive them, meaning that they might have to read through all of them
• When a client receives messages, they become temporarily locked in order to prevent other clients from reading them (this behavior can be overridden)
• There is no concept of a "reply message" in SQS like there is in JMS. To send a reply, you must use a different queue or a different mechanism

For a kv store, you should use something like Amazon SimpleDB or DynamoDB.

Answer 2

SQS is probably not the right tool for this use case. You cannot "retrieve messages by key", you can only "retrieve messages", the message you receive being entirely up to whatever SQS wants to give you.

I would take a look at DynamoDB or S3 depending on your requirements. DynamoDB for low-latency, predictable throughput of structured data and S3 if you don't have requirements for these features.

Answer 3

SQS is not right choice for this key pair storage...use dynamo DB with SQS...

Answer 4

You can also combine SQS and S3, where you can store the url of each individual S3 object entry in the SQS queue as reference.