Well, I got inspired a lot by the comments from you guys and I came up with a solution that compress the HTML content using zlib
and POST the data to API server, on the Flask API server side, I extract the data and push to mongodb for storage.
Here is the part that might save some future headache.
Client Side:
myinput = "http://www.exmaple.com/001"
myoutput = "<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" ... /html>"
result = {'myinput':myinput, 'myoutput': myoutput}
data = zlib.compress(str(result))
opener.open("www.host.com/senddata", data)
Server Side:
@app.route('/contribute', methods=['POST'])
def contribute():
try:
data = request.stream.read()
result = eval(zlib.decompress(data))
db.result.insert(result)
except:
print sys.exc_info()
pass
return 'OK'
Result in mongodb:
{
"_id" : ObjectId("534e0d346a1b7a0e48ff9076"),
"myoutput" : "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" ... /html>",
"myinput" : "http://www.exmaple.com/001"
}
(Note: As you have noticed, the final version in mongo somehow escaped all the sensible characters by putting a slash in front of them, like double quote, not sure how to change it back.)
There were some discussions about retrieving binary data in flask. Like here.
So you don't have to mess up with the header if you read from request.stream directly
.
Thanks!