Bulk querying the status of the async Copy Blob Azure API

Question

A while ago, I've asked a question on how to copy a large number of blobs from one account to another. The best solution I found was to use the new Copy Blob API that allows to copy blobs between accounts without downloading them.

I have since then tested the new API (which is now available in the Azure SDK, at least in the latest gitHub version of it) and my benchmarks have shown a significant improvement - while it took 1.5 minutes to copy a 90MB blob the "old way" (i.e. download, then upload), it took only 10 seconds to copy via Copy Blob. That's the good news.

Now, the bad news: the Copy Blob API is async, which means that it returns immediately, and I need to poll the destination blob's properties to find out when the copy finished. This is great, in principle, but it's extremely inefficient when I have thousands of blobs to copy.

So my question is: what is a good, efficient way of finding out when the copy finished when copying thousands (in my case, over 600,000) blobs from one storage account into another using the Copy Blob API?

Here is some code, super-slow, that can find out the completed percentage:

        foreach (ICloudBlob blob in cloudBlobContainer.ListBlobs())
        {
            totalCount++;

            // TOO SLOW!
            ICloudBlob blobDetails = cloudBlobContainer.GetBlobReferenceFromServer(blob.Name);

            if (blobDetails.CopyState.Status == CopyStatus.Success)
            {
                successCount++;
            }
        }

Answer 1

CloudBlobContainer.ListBlobs method takes an argument named blobListingDetails that describes what to include in the listing. If you specify BlobListingDetails.Copy, listing will also include copy properties.

Please refer to the MSDN article for more information.