Upload File with brackets ([ & ]) in the name

Question

I'm moving a ClickOnce install from a regular web server to Azure Blob storage and have a problem with some of the files. The filenames contains [ ] and CloudBlob.UploadFile fails with an exception:

Microsoft.WindowsAzure.Storageclient.StorageException:
Error accessing blob storage: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

The code has been used for a while and only fails on files with [ ] in the name so I don't believe that it is an "authentication failure". In this particular case, this is the seventh file being uploaded in a loop. I found this link on MSDN about valid file names and this on stack overflow which both show problems with square brackets in URL's and reference UrlEncode. I added in a call to UrlEncode and that did not help. The container is created with public access since we use it to support customer downloads of our software. We have been hosting a "test" install in another container and have not had permission problems accessing that either.

I can upload the file with no name changes and then rename the file to add the "path" using newdesic's Azure Storage Explorer tool so what is that doing that I am not doing?

Answer 1

I see you're using the 1.7 SDK. This is a small encoding issue with the SDK which is also present in v2.0. Let's see what happens.

No encoding

account.CreateCloudBlobClient()
       .GetContainerReference("temp")
       .GetBlobReference("abc[]def.txt")
       .UploadFile("myfile.txt");

If you don't encode the blob name, you'll end up with a request to the following URL which is causing the authentication exception:

http://account.blob.core.windows.net/temp/abc[]def.txt

The is because the SDK uses Uri.EscapeUriString internally to encode your string, but this doesn't take into account square brackets.

Encoding

Then you would expect the following to do the trick:

account.CreateCloudBlobClient()
       .GetContainerReference("temp")
       .GetBlobReference(HttpUtility.UrlEncode("abc[]def.txt"))
       .UploadFile("myfile.txt");

The issue here is that you'll end up with this url:

http://account.blob.core.windows.net/temp/abc%255b%255ddef.txt

So what's happening here? Calling HttpUtility.UrlEncode turns abc[]def.txt to abc%5B%5Ddef.txt, which is correct. But internally, the SDK will encode this string again which results in abc%255b%255ddef.txt, which isn't what you want.

Workaround

The only way to apply encoding which takes square brackets into accounts is by using a small workaround. If you pass the full URL to the GetBlobReference method, the SDK assumes you did all the encoding yourself:

var container = account.CreateCloudBlobClient().GetContainerReference("temp");
var blob = container.GetBlobReference(String.Format("{0}/{1}", 
                container.Uri, System.Web.HttpUtility.UrlEncode("abc[]def.txt")));
blob.UploadFile("myfile.txt");

This results in a correctly encoded URL:

http://account.blob.core.windows.net/temp/abc%5b%5ddef.txt

And if you use a tool like CloudXplorer, you'll see the blob with the correct filename:

Answer 2

There are two known breaks in the Uri class in .Net 4.5

• ‘[‘,’]’ characters are no longer escaped

• ‘\’ character is now escaped as %5C

This is causing an authentication when the server attempts to validate the signature of the request as the canonicalized string is now different on the client and the server.

There are a few workarounds clients can use while this issue is present. The correct solution will depend on your specific application and requirements. Avoid the ‘[‘,’]’, or ‘\’ characters in resource names By simply avoiding these characters all together you will be able to avoid the issue described above.

Target .Net 4.0

Currently the recommendation is for clients to simply continue to target their applications to .Net 4.0 while a full solution is being investigated. Note, since .Net 4.5 is an in place upgrade clients can still take advantage of some performance improvements in the GC etc, without specifically targeting the .Net 4.5 profile. For Windows RT developers this is not an option and you will therefore require the workarounds detailed below.

Pre-Escape Data

If possible a client can pre- escape the data or replace the given characters with non-affected ones. This is why the workaround above is working.