Ignore pycrypto in buildout $app/dist/ folder when deploying to Google App Engine

Question

I am running into some difficulties deploying an app that I have been developing locally using dev_appserver 1.7.6 in a virtualenv with buildout. When I deploy, my local dist folder is also deployed which causes conflicts with the app engine provided libraries

My app depends on pycrypto so in order to run it locally I have added pycrypto to my buildout.cfg. This installs pycrypto in $app/dist/Crypto. Everything runs fine and dandy locally but when I deploy the $app/dist folder is also deployed. App Engine then complains with an ImportError when trying to import DES3

File "/path/to/my/app/dist/Crypto/Cipher/DES3.py", line 67, in <module>
    from Crypto.Cipher import _DES3
ImportError: dynamic module does not define init function (init_DES3)

You can see from the error log that it is trying to use the pycrypto in $app/dist instead of the google provided version.

How can I go about using pycrypto in virtualenv and still deploy to Google App Engine without having to comment/uncomment dist/Crypto from my skip_files section?

I've only been working python for a few months now so I assume I am missing something obvious due to my lack of experience with the environment.

Answer 1

As it turns out my local setup was very much broken due to my misunderstanding of buildout / virtualenv.

The solution was to create two separate parts with recipe = appfy.recipe.gae:app_lib in my buildout.cfg. One part is for items that app engine includes when you deploy but not in the local app server. This uses a lib-directory outside of the main application path. The second part is also a recipe = appfy.recipe.gae:app_lib but it uses a lib-directory inside of the folder of code being deployed to app engine.