Uploading a huge file from ec2 to s3 fails

Question

I'm trying to upload 160 Gb file from ec2 to s3 using

s3cmd put --continue-put FILE s3://bucket/FILE

but every time uploading interrupts with the message:

FILE -> s3://bucket/FILE [part 10001 of 10538, 15MB] 8192 of 15728640 0% in 1s 6.01 kB/s  failed

ERROR: Upload of 'FILE' part 10001 failed. Aborting multipart upload.
ERROR: Upload of 'FILE' failed too many times. Skipping that file.

The target bucket does exist.

What is the issue's reason?

Are there any other ways to upload the file?

Thanks.

Answer 1

You can have up to 10000 upload parts per object, so it fails on part 10001. Using larger parts may solve the issue.

Answer 2

"huge"---is it 10s or 100s of GBs? s3 limits the object size to 5GB and uploading may fail if it exceeds the size limitation.