Github ran into this same issue when they moved to HTTPS everywhere and detailed it in their blog: https://github.com/blog/743-sidejack-prevention-phase-3-ssl-proxied-assets
Their solution was to create a proxy server which they open sourced as https://github.com/atmos/camo To address the same concerns about abuse of the proxy it is deployed with a shared secret with the application server. Integrating this would a Django project would be straight forward as you would just need to generate the digest from the shared key for the given image url.