python: download file and send it to online storage in realtime

Question

I want to download file to my server and automatically send it to online storage(minus or dropbox) via minus or dropbox API, without saving the downloaded file in my server. So, its like streaming or pipe the HTTP connection. Right now im using minus.com API, but its require file object or local file as parameter. I can't figure out how to convert http response to file object.

It is possible to do this? if possible, how?

concept :

FILE_ON_ANOTHER_SERVER ----(http)---> MY_SERVER ----(http)----> ONLINE_STORAGE

thanks

Answer 1

You can get the data from a response via the read() method

response = urllib2.urlopen(request)

data = response.read()

The variable data has the binary data from the response.

Now you can create a StringIO Object which handles the data as a file like object.

import StringIO
datastream = StringIO.StringIO()
datastream.write(data)
datastream.seek(0)
#create dropbox client
client.put_file('/test', datastream)

Answer 2

urllib2.urlopen(url) will return a file-like object. Can you pass that directly to your minus api? See the urllib2 docs at http://docs.python.org/library/urllib2