practical solution for dynamodb's max item length (64k) and no-sql's lack of table joins

Question

First, some background context that I think may be very relevant:

I'm proofing dynanodb as a viable solution for an upcoming app. This is a learning experiment to familiarize myself with the dynamodb offering. My platform is amazon EC2 (Linux), node.js with the dynode library.

My question came along when considering one-to-many relationships (e.g. tags, keywords, categories, etc) in the context of no-sql, where it's not possible to join tables. At first it seemed a no-brainer: just store them as an attribute list. But then I read about the 64KB limit.

so my question:

An item, including it's attributes, (i.e. a db row with all its columns) in DynamoDB cannot exceed a total of 64KB binary size. So how do I go about checking the data to make sure it will not violate that length?

Obviously numbers are straightforward, but "Unicode with UTF8 binary encoding" strings are going to be a challenge.

The second part of this question might be obvious to the no-sql ninjas: how do I avoid bumping into this limitation while easing the burden of not being able to join tables? A lengthy answer is likely beyond the scope of the Q&A format, but if anyone has comments or could offer alternatives I should be exploring while solving the 64k length, I would be grateful.

Best, and thanks in advance for any and all suggestions.

Answer 1

I concur with the comment by @bcoates. It really does depend on your use case.

I have several implementations (in SimpleDB, but same idea) that use an EntityRelationship table to model joins, with bits of relevant data duplicated to those tables. With DynamoDB, you could make good use of a range key for implementing tags, categories, etc...

Also, I have used attribute spanning to overcome the attribute size limitations in SimpleDB (DynamoDB, the same could be accomplished). Multiple attributes, combined after fetching in order to reconstitute the entire record. (Range Key tables would be helpful here too)

Simol (for C#) has a good implementation for SimpleDB (http://simol.codeplex.com), and I know that the project owner is working on a similar implentation for DynamoDB - you may be able to gleen some insight from that project.

Answer 2

If anyone is coming across this late in the day, the maximum item size is now 400kb rather than 64kb