Why calling Python script from setup.py invokes Python shell instead?

Question

I try to call a Python script from setup.py during installation, using a custom install:

class CustomInstall(install):
    def run(self):
        install.run(self)

        ...

        p = subprocess.Popen(
            [sys.executable, 'demo_package/deploy_database.py'],
            shell=True,
            stdout=subprocess.PIPE,
            cwd=os.path.join(self.install_lib, 'demo_package'))

        out, err = p.communicate()

setup(..., cmdclass=dict(install=CustomInstall))

When deploying the package on a Ubuntu machine, the process, instead of executing deploy_database.py, shows nothing. When I stop it manually with Ctrl+C, the output seems to indicate that instead of actually running deploy_database.py, it simply starts Python:

^CDownloading/unpacking PypiPackagesMonitoring
  Downloading demo-1.0.64.zip  
  Running setup.py egg_info for package demo

Installing collected packages: demo
  Running setup.py install for demo

    Python 3.3.2+ (default, Oct  9 2013, 14:50:09)
    [GCC 4.8.1 on linux
    Type "help", "copyright", "credits" or "license" for more information.
Cleaning up...
Operation cancelled by the user
Storing complete log in /home/.../.pip/pip.log

What is wrong in the way I call a Python script? How should I do it instead?

Answer 1

shell=True is not needed with [sys.executable, 'demo_package/deploy_database.py'], and causes problems. The way you're doing it (if you omit shell=True) is the preferred way, because it bypasses the shell.

shell=True makes Popen hand over the passed command to a shell (e.g. /bin/bash, depending on the shell configured for the current user). Only the first command passed in the list is executed, the other elements of the list are passed as arguments to the shell itself (from the docs):

On Unix with shell=True […] If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself.

shell=True is a bit dangerous, but also useful in some cases. See the documentation for details.