How to cancel a long-running Amazon Glacier file upload operation in .NET C#

Question

I am new to Amazon AWS and Glacier. I am trying to write a WPF Windows-based C# client that uploads my archived backup data to the glacier cloud. However, the API reference don't seem to offer a cancel command. Only upload, download, list.

What I'm trying to do is run each upload operation (which can take 1 hour or more with large files) asynchronously using TPL. However I want the upload to be cancellable, which .NET 4.5 would support nicely, but the Amazon API does not.

Is there a way to do that anyway?

Thanks.

Answer 1

The recommended way to handle your scenario in Amazon Glacier (and Amazon S3 as well btw.) is to Upload archives in parts via Multipart Upload, see Uploading an Archive in Amazon Glacier:

Depending on the size of the data you are uploading, Amazon Glacier offers the following options:

• Uploading Large Archives in Parts (Multipart Upload) — In a single operation, you can upload archives from 1 byte to up to 4 GB in size. However, we encourage Amazon Glacier customers to use Multipart Upload to upload archives greater than 100 MB. [...] [emphasis mine]

• Upload archives in parts — Using the Multipart upload API you can upload large archives, up to about 40,000 GB (10,000 * 4 GB).

Uploading Large Archives in Parts (Multipart Upload) provides the details on the latter, specifically regarding Complete (or Abort) Multipart Upload:

After uploading all the archive parts, you use the complete operation. [...]

If you abort a multipart upload, you cannot upload any more parts using that multipart upload ID. All storage consumed by any parts associated with the aborted multipart upload is freed. If any part uploads were in-progress, they can still succeed or fail even after you abort. [emphasis mine]

So you still can't abort the uploads of parts that are in progress as such, thus the key for the desired user experience and/or network bandwidth reduction is choosing a small enough part size.

Depending on your use case you might also want to check into List Multipart Uploads (GET multipart-uploads):

This multipart upload operation lists in-progress multipart uploads for the specified vault. An in-progress multipart upload is a multipart upload that has been initiated by an Initiate Multipart Upload (POST multipart-uploads) request, but has not yet been completed or aborted. [...]