Does the MongoDB Shard Key Need to be Unique?

Question

I'm struggling with this a bit, having come from SQL environment where we sharded on user id. We had multiple tables with the user_id added to it, so that we could shard. If there were 100 records in one table, all for the same user, all 100 would end up in the same shard.

So, we have our first mongo database, that we'd like to be able to shard if necessary. There is a user collecton, which we intend to shard by _id. No problem there. But there is a second collection, which could have hundreds of docs for one user in it. We'd like those to go to the same shard, (doesn't have to be the same as the user doc, as they get retrieved seperately, but they do get retrieved in chunks by user.) But it looks like, if we shard by user_id, which has been added to the second collection, this is not sufficient, the shard key needs to be unique, so every time we did a lookup, we'd be going across all shards. This is not optimal. So, does it need to be completely unique, as in, only one such record per collection?

The documentation indicates that we have a problem. I'm hoping I'm not understanding.

Answer 1

No the sharding key should not necessarily be unique (although it can be). For example the following sharding key {userID : 1, countryID : 1} is absolutely valid for a collection which has fields (note that two elements has the same userID and countryID):

{userID : 4, countryID : 5},
{userID : 4, countryID : 9},
{userID : 1, countryID : 5},
{userID : 2, countryID : 3},
{userID : 4, countryID : 5},
{userID : 5, countryID : 4}

As far as I remember, mongo does not allow you to create sharding key only on missing key (and there was a bug about it). And also you can not change the value of the sharding key after this.

But you can even have the same field {a : 5} across all collection and have it as a sharding key (this would be totally stupid, but you can do this).

You have to be kind of careful when choosing your sharding key, because it is big pain in the ass of changing it afterwards. So the good thing is too read for some time how to choose a sharding key.