pip with fake computer architecture (64 bit instead 32 bit)

Question

I host a Django website on a server where I have only FTP access. The server has a 64 bit architecture, my computer has a 32 bit architecture. For some modules, like MySQL-python, the architecture matters.

Is there a way to build a site-packages directory from my requirements.txt on my personal computer which is suitable for the server? I think it would be necessary to somehow fake the architecture for pip.

And ... Yes. I have access to some 64 bit machines. But for a good workflow it would better to have everything on my personal computer.

Answer 1

What you're asking for is much harder than you think.

Many libraries—including MySQLdb—contain C extension code that must be compiled for the target system. If you compile them to run on your system, they will not work on the target system. That means you need a cross-compiler.

They also need to be linked to the Python libraries that exist on the target system, not the ones on your system. And they will almost always need the headers and configuration files that go with those libraries.

Most such libraries have other dependencies—in this case, you'll almost certainly need the libmysql from the target system, not the one on your system, and of course the headers and config files that go with that.

And, if you want to use pip, you will need a distutils/setuptools configuration that selects the right toolchain, libraries, etc. to build for.

If you do all that, then you can build a site-packages for another system's Python.

If you Google "cross compile python module", you'll find blogs like this one that will give you some guidance if you've never done anything like this before. Of course, as it typical with blog posts describing things that aren't easy, they go out of date pretty quickly (the first three I looked at all start off with installing distribute to replace setuptools, which you definitely do not want to do anymore), so you'll have to really understand what they're doing and be able to debug the process yourself, not just follow them blindly.

---

If you don't want to or can't do that yourself, then you can't do what you want. But there's a much simpler solution anyway.

Just set up a development system for the server. If you're running on 64-bit hardware, you can dual-boot, or use virtualization tools like virtualbox or VMware to run a 64-bit linux on top of the 32-bit one. Install the same distro and the same packages that are on the target server, and the accompanying -dev/-devel/whatever packages, and build there.

If your hardware were actually 32-bit, or your target were, say, OS X rather than Linux, or the target environment took up so much disk space just to set up that it wouldn't fit on your laptop, then this would obviously not be so easy. But none of those are true, so it is that easy.

Answer 2

Do local development in a VM

Use virtualbox and make your guest OS be 64-bit.

You can still edit locally (through sharing folders), but deploy to the virtual machine that matches the architecture. This will save you much pain down the road.

The biggest reason for doing this is that many of the libraries have to be compiled for the architecture. Pure python won't be a problem.

Use virtualenv on the remote host

Additionally, when you install the packages and deploy your code, you should use a virtual environment instead of site-packages. Any admin on the box can nuke your glorious site-packages away with system updates. We don't want that!