S3 Java- Multipart upload using a presigned URL?

Question

I have a separate services that is managing files and s3 authentication. It produces presigned URLs, which I am able to use in other services to upload (and download) files.

I would like to take advantage of the Multipart upload sdk- currently the 'uploadToUrl' method seems to spend most of its time on getResponseCode, so it's difficult to provide user feedback. Also, the multipart upload seems much faster in my testing.

Ideally, I'd like to be able to create some AWSCredentials using a presigned URL instead of a secret key / access key for temporary use. Is that just a pipe dream?

//s3 service
public URL getUrl(String bucketName, String objectKey, Date expiration, AmazonS3 s3Client, HttpMethod method, String contentType) {
    GeneratePresignedUrlRequest generatePresignedUrlRequest;

    generatePresignedUrlRequest = new GeneratePresignedUrlRequest(bucketName, objectKey);
    generatePresignedUrlRequest.setMethod(method);
    generatePresignedUrlRequest.setExpiration(expiration);
    generatePresignedUrlRequest.setContentType(contentType);

    URL s = s3Client.generatePresignedUrl(generatePresignedUrlRequest);
    System.out.println(String.format("Generated Presigned URL: %n %S", s.toString()));
    return s;
}

//Upload service
Override
public void uploadToUrl(URL url, File file) {

    HttpURLConnection connection;
    try {
        InputStream inputStream = new FileInputStream(file);
        connection = (HttpURLConnection) url.openConnection();
        connection.setDoOutput(true);
        connection.setRequestMethod("PUT");
        OutputStream out =
                connection.getOutputStream();

        byte[] buf = new byte[1024];
        int count;
        int total = 0;
        long fileSize = file.length();

        while ((count =inputStream.read(buf)) != -1)
        {
            if (Thread.interrupted())
            {
                throw new InterruptedException();
            }
            out.write(buf, 0, count);
            total += count;
            int pctComplete = new Double(new Double(total) / new Double(fileSize) * 100).intValue();

            System.out.print("\r");
            System.out.print(String.format("PCT Complete: %d", pctComplete));
        }
        System.out.println();
        out.close();
        inputStream.close();

        System.out.println("Finishing...");
        int responseCode = connection.getResponseCode();

        if (responseCode == 200) {
            System.out.printf("Successfully uploaded.");
        }
    } catch (IOException e) {
        e.printStackTrace();
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}

Answer 1

For now I am exploring returning temporary credentials returned by my trusted code instead of a signed url.

http://docs.aws.amazon.com/AmazonS3/latest/dev/AuthUsingTempSessionTokenJava.html

Answer 2

A few years later, but digging around in the AWS Java SDK reveals that adding the following to GeneratePresignedUrlRequest works pretty well:

AmazonS3Client amazonS3Client = /* ... */;
GeneratePresignedUrlRequest request = /* ... */;

// the following are required to trigger the multipart upload API
request.addRequestParameter("uploadId", uploadIdentifier);
request.addRequestParameter("partNumber", Integer.toString(partNumber));

// the following may be optional but are recommended to validate data integrity during upload
request.putCustomRequestHeader(Headers.CONTENT_MD5, md5Hash);
request.putCustomRequestHeader(Headers.CONTENT_LENGTH, Long.toString(contentLength));

URL presignedURL = amazonS3Client.generatePresignedUrl(request);

(I haven't dug deeply enough to determine whether CONTENT_MD5 or CONTENT_LENGTH are required.)

Answer 3

with PHP, you can

$command = $this->s3client->getCommand ('CreateMultipartUpload', array (
    'Bucket'    => $this->rootBucket,
    'Key'       => $objectName
));
$signedUrl = $command->createPresignedUrl ('+5 minutes');

But I found no way so far how to acheive this with Java. For a single PUT (or GET) operation, one can use generatePresignedUrl, but I wouldn't know how to apply this to multipart upload like with the PHP getCommand ('CreateMultipartUpload'/'UploadPart'/'CompleteMultipartUpload') methods.