How can store a file on Google Storage from URL on Google App Engine?

Question

I want to create a service on Google App Engine (Python) that will receive a URL of an image and store it on Google Storage. I managed to upload from a local file using boto or gsutil command line, but not by retrieving the file via URL. I tried doing it using the HTTP requests (PUT) and I'm getting error responses for wrong signatures. Obviously I'm doing something wrong, but unfortunately I have no idea where.

So my question is: How can I retrieve a file from a URL and store it on Google Storage using Python for Google App Angine?

Here is what I've done (using another answer):

class ImportPhoto(webapp.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        srow = self.response.out.write
        url = self.request.get('url')
        srow('URL: %s\n' % (url))
        image_response = urlfetch.fetch(url)
        m = md5.md5()
        m.update(image_response.content)
        hash = m.hexdigest()
        time = "%s" % datetime.utcnow().strftime("%a, %d %b %Y %H:%M:%S GMT")
        str_to_sig = "PUT\n" + hash + "\n\n" + 
                      time + "\nx-goog-acl:public-read\n/lipis/8418.png"
        sig = base64.b64encode(hmac.new(
                                  config_credentials.GS_SECRET_ACCESS_KEY,
                                  str_to_sig, hashlib.sha1).digest())
        total = len(image_response.content) 
        srow('Size: %d bytes\n' % (total))

        header = {"Date": time,
                  "x-goog-acl": "public-read",
                  "Content-MD5": hash,
                  'Content-Length': total,
                  'Authorization': "GOOG1 %s:%s" % 
                                    (config_credentials.GS_ACCESS_KEY_ID, sig)}

        conn = httplib.HTTPConnection("lipis.commondatastorage.googleapis.com")
        conn.set_debuglevel(2)

        conn.putrequest('PUT', "/8418.png")
        for h in header:
            conn.putheader(h, header[h])
        conn.endheaders()
        conn.send(image_response.content + '\r\n')
        res = conn.getresponse()

        srow('\n\n%d: %s\n' % (res.status, res.reason))
        data = res.read()
        srow(data)
        conn.close()

And I'm getting as a response:

URL: https://stackoverflow.com/users/flair/8418.png
Size: 9605 bytes

400: Bad Request
<?xml version='1.0' encoding='UTF-8'?><Error><Code>BadDigest</Code><Message>The Content-MD5 you specified did not match what we received.</Message><Details>lipis/hello.jpg</Details></Error>

Answer 1

Have you read the docs on how to sign requests? The string to sign must include the Content-MD5, Content-Type and Date headers, in addition to the custom headers and the resource path.

Answer 2

Content-MD5 header is optional for PUT requests. Try leaving this out for a test.

Also, required headers are Authorization, Date and Host. It seems that your request is missing Host header.