You can have up to 10000 upload parts per object, so it fails on part 10001. Using larger parts may solve the issue.
Uploading a huge file from ec2 to s3 fails
-
10-07-2023 - |
문제
I'm trying to upload 160 Gb file from ec2 to s3 using
s3cmd put --continue-put FILE s3://bucket/FILE
but every time uploading interrupts with the message:
FILE -> s3://bucket/FILE [part 10001 of 10538, 15MB] 8192 of 15728640 0% in 1s 6.01 kB/s failed
ERROR: Upload of 'FILE' part 10001 failed. Aborting multipart upload.
ERROR: Upload of 'FILE' failed too many times. Skipping that file.
The target bucket does exist.
What is the issue's reason?
Are there any other ways to upload the file?
Thanks.
해결책
다른 팁
"huge"---is it 10s or 100s of GBs? s3 limits the object size to 5GB and uploading may fail if it exceeds the size limitation.
제휴하지 않습니다 StackOverflow