sequentiual numbering in the cloud

Question

Ok so a simple task such as generating a sequential number has caused us an issue in the cloud. Where you have more than one server it gets harder and harder to guarantee that the allocated number between servers are not clashing.

We are using Azure servers if it helps.

We thought about using the app cache but you cannot guarantee it will be updated between servers.

We are limited to using: a SQL table with an identity column or some peer to peer method between servers or use a blob store and utalise the locks to store the nost upto date number. (this could have scaling issues)

I just wondered of anyone has an idea of a solution to resolve this? Surely its a simple problem and must have been solved by now.

Answer 1

If you can live with a use-case where sometimes the numbers you get from this central location are not always sequential (but guaranteed to be unique) I would suggest considering the following pattern. I've helped an large e-commerce client implement this since they needed unique int PK's to synchronize back to premise:

Create a queue and create a small always-running process that populates this queue with sequential integers (this process should remember which number it generated last and keep replenishing the pool with more numbers once the queue gets close to be empty)

Now, you can have your code first poll the next number from the queue, delete it from the queue and then attempt to save it into the SQL Azure database. In case of failure, all you'll have is a "hole" in your sequential numbers. In scenarios with frequent inserts, you may be saving things out of order to the database (two processes poll from queue, one polls first but saves last, the PK's saved to the database are not sequential anymore)

The biggest downside is that you now have to maintain/monitor a process that replenishes the pool of PK's.

Answer 2

After read this, I would not trust on identity column.

I think the best way is before insert, get the last stored id and increment it by one. (programatically). Another option is create a trigger, but it could be a mass if you'll receive a lot of concurrent requests on DB or if your table have millions of records.

create trigger trigger_name
on table_name
after insert
as 
declare @seq int
set @seq = (select max(id) + 1 from table_name)
update  table_name
    set  table_name.id =  @seq
    from  table_name 
    inner join inserted
    on  table_name.id = inserted.id

More info:

http://msdn.microsoft.com/en-us/library/windowsazure/ee336242.aspx

Answer 3

If you're worried about scaling the number generation when using blobs, then you can use the SnowMaker library which is available on GitHub and Nuget. It gets around the scale problem by retrieving blocks of ids into a local cache. This guarantees that the Ids are unique, but not necessarily sequential if you have more than one server. I'm not sure if that would achieve what you're after.