Azure CloudPageBlob Create - how to implement CreateIfNotExist

Question

I am using Azure SDK version 2.2. I am having problem, that in my code potentially multiple machines call this code

CloudPageBlob _pageBlob = .....
try
{
    _pageBlob.FetchAttributes();             
}
catch
{
    //container does not exist
    _pageBlob.Create(AllocatedBlobSize);
}

I have tested it, and my conclusion is that multiple calls of _pageBlob.Create(AllocatedBlobSize); clears previous data. How to ensure that only one machine wins and creates CloudPageBlob? (I want others to fail) I was expecting that multiple Create calls would result in exception but that is not what is happening :(.

EDIT: In our architecture there is one leader and X followers. I am using CloudPageBlob + ETAG to elect new leader (only one machine will be able to write it's metadata to CloudPageBlob = new leader). But there is extreme case, when this cloud page blob can get corrupted + leader dies(or we are starting and there is no leader yet). In this case, I need to create new CloudPageBlob (normaly leader creates it) => each machine is trying to create it and write it's metadata to it (everyone want to be leader). In theory, if 2 machine were executing this code. Machine A calls _pageBlob.Create(), no error, writes metadata thinks it's leader. Machine B call _pageBlob.Create(), no error, metadata of machine A(+ potentially more data) are rewritten by machine B. I want machine B to fail :). The biggest problem is that I am loosing data.

Answer 1

Solution I have found is to use AccessCondition.IfNotModifiedSince.

var timeOffset = new DateTimeOffset(new DateTime(1, 1, 1), TimeSpan.FromMilliseconds(0));
var cond = AccessCondition.GenerateIfNotModifiedSinceCondition(timeOffset);
var options = new BlobRequestOptions ();
pageBlob.Create(AllocatedBlobSize, cond, options);

Now If I call CloudPageBlob.Create() for second time with same condition, second call fails. What is exactly I wanted. Thanks all for help.

Answer 2

In this scenario, you'd better implement a queue to act as a flag. You'll create and send a message to a queue as a request to create your page blob. So, only one machine will dequeue the message and then call your pageBlob create method.

Answer 3

Another approach is to go with blob leasing. You would have a block blob and let all instances try to acquire a lease on that blob. Only one instance will be able to succeed. The instance which succeeds will call the code to create the blob. Depending on when in your application life cycle you're invoking the operation, you may still have to use the code above so that if a blob already exists, you don't recreate it.

CloudPageBlob _pageBlob = .....
try
{
    acquireBlobLease();
    try
    {
        _pageBlob.FetchAttributes();             
    }
    catch
    {
        //container does not exist
        _pageBlob.Create(AllocatedBlobSize);
    }
}
catch
{
}

Answer 4

I call this a "traffic cop" pattern but I've also heard it called a "leader election" pattern (http://msdn.microsoft.com/en-us/library/dn568104.aspx). The best approach is to use a blob lease. I've actually created some sample c# code you can use to help implement this: http://brentdacodemonkey.wordpress.com/2012/07/27/the-traffic-cop-pattern/

A queue works well as long as there is a process that will insert a new queue message if the current leader "dies". You've also already pointed out the issue with the meta-data approach. The blob lease fixes this by ensuring that only one instance will get the lease and if that instance dies, the lease will be released and a new leader can be elected.

As for your original example, I would recommend instead creating the blob as part of the deployment or even pre-deployment. This eliminates the need to manage the blob creation as part of the role instance (fewer operations against storage). If you're not comfortable with that route, then flip your code a bit and try to do the create, trap for the exception if it already exists, and then continue processing with trying to do the update (which will require a lease anyway to prevent two instances from updating the blob with their meta-data at the same time and thus thinking they are both leaders).