Any way to serve user uploaded images from inside zips with python?

Question

So I am making a site where users can upload galleries of images(usually dozens of images per gallery), and they should be compressed into a zip file(by the user) before upload. I notice how (at least on Linux) I can click on a compressed folder and view the images inside it without extracting it. Is it possible to serve images from inside zipfiles with python like this, as decompressing is CPU intensive?

Answer 1

You can read the contents of zip files using Python using the zipfile module which would allow you to decompress and serve files on the fly. Whether this is actually what you want to do is another question, since, as you mention, decompressing the file will be more CPU intensive than just serving the file directly from the file system.

I am guessing that users are uploading files in a zip so that they can upload multiple images at once. In this case it might be better to just decompress the zip file after it has been uploaded. You can then decompress the file into a location from which you can serve the files as static files using a webserver.

Sample application

The following code shows a sample Flask application that demonstrates how to serve up images directly from zip files. It cuts a lot of corners so don't use this directly in a production app!

To test the app, install flask (probably into a virtualenv) and run the code:

virtualenv env
env/bin/pip install flask
env/bin/python sample_flask_application.py

Make sure that you have a directory called galleries at the same level as the flask application and that you have a few .zip files containing jpegs in the galleries directory.

import os
import zipfile

from flask import Flask
from flask import Response

app = Flask(__name__)

gallery_path = 'galleries'

@app.route('/')
def index():
    html = '<a href="{0}">{0}</a>'
    gallery_zips = os.listdir(gallery_path)
    gallery_names = [os.path.splitext(zfile)[0] for zfile in gallery_zips]
    galleries = [html.format(gallery) for gallery in gallery_names]
    return '<br>'.join(galleries)


@app.route('/<gallery>')
def gallery(gallery):
    zip_path = os.path.join(gallery_path, gallery + '.zip')
    html = '<a href="{0}/{1}">{1}</a>'
    with zipfile.ZipFile(zip_path, 'r') as zfp:
        images = zfp.namelist()
        image_list = [html.format(gallery, image) for image in images]
        return '<br>'.join(image_list)


@app.route('/<gallery>/<path:image>')
def image(gallery, image):
    zip_path = os.path.join(gallery_path, gallery + '.zip')
    with zipfile.ZipFile(zip_path) as zfp:
        image_fp = zfp.open(image, 'r')
        return Response(image_fp, mimetype='image/jpeg')

if __name__ == '__main__':
    app.run(debug=True)

Possible improvements

• Set expires and other things that might help cache the image files in the image function
• Use a cache of decompressed images
  • When serving an image, first check if the image is in the cache location
  • If not, decompress it to the cache location
  • Then use send_file to serve the file to client
• Cache the list of files contained in each zipfile
• Generate a 404 if the image is not actually in the zip file!

Conclusions

The above code will work, especially if you are on a low traffic site. If a few people look at a few images a day, then you'll be fine. If you optimise the above code a bit (expires, caching) then you can deal with more traffic still. However if you want the best performance then you are looking at something that decompresses the zip files when they are uploaded and serves them up using a webserver directly (nginx, apache, or whatever you are using). When you get thousands of visitors from all over the world then you can start looking at content delivery networks as well.