python setuptools install_requires is ignored when overriding cmdclass

Question

I have a setup.py that looks like this:

from setuptools import setup
from subprocess import call
from setuptools.command.install import install

class MyInstall(install):
    def run(self):
        call(["pip install -r requirements.txt --no-clean"], shell=True)
        install.run(self)

setup(
    author='Attila Zseder',
    version='0.1',
    name='entity_extractor',
    packages=['...'],
    install_requires=['DAWG', 'mrjob', 'cchardet'],
    package_dir={'': 'modules'},
    scripts=['...'],
    cmdclass={'install': MyInstall},
)

I need MyInstall because I want to install some libraries from github and I didn't want to use dependency_links option, because it's discouraged (for example here), so I can do this with requirements.txt.

When I install this package with pip, everything is working fine, but for some reasons I have to solve this in a way that it also works with pure python setup.py install. And it doesn't.

When overriding cmdclass in setup() with my own class, install_requires seems to be ignored. As soon as I comment out that line, those packages are being installed.

I know that install_requires is not supported for example in distutils (if I remember well), but it is in setuptools. And then cmdclass wouldn't have any effect on install_requires.

I googled this problem for hours, found a lot of kind of related answers on stackoverflow, but not for this particular problem.

With putting every needed package to requirements.txt, everything's working fine, but I would like to understand why this is happening. Thanks!

Answer 1

The same problem just happened to me. It somehow seems like something triggers setuptools to do an 'old-style install' with distutils, which indeed does not support install_requires.

You call install.run(self) which calls run(self) in setuptools/setuptools/command/install.py, line 51-74

https://bitbucket.org/pypa/setuptools/src/8e8c50925f18eafb7e66fe020aa91a85b9a4b122/setuptools/command/install.py?at=default

def run(self):
    # Explicit request for old-style install?  Just do it
    if self.old_and_unmanageable or self.single_version_externally_managed:
        return _install.run(self)

    # Attempt to detect whether we were called from setup() or by another
    # command.  If we were called by setup(), our caller will be the
    # 'run_command' method in 'distutils.dist', and *its* caller will be
    # the 'run_commands' method.  If we were called any other way, our
    # immediate caller *might* be 'run_command', but it won't have been
    # called by 'run_commands'.  This is slightly kludgy, but seems to
    # work.
    #
    caller = sys._getframe(2)
    caller_module = caller.f_globals.get('__name__','')
    caller_name = caller.f_code.co_name

    if caller_module != 'distutils.dist' or caller_name!='run_commands':
        # We weren't called from the command line or setup(), so we
        # should run in backward-compatibility mode to support bdist_*
        # commands.
        _install.run(self)
    else:
        self.do_egg_install()

I'm not sure whether this behaviour is intended, but replacing

install.run(self)

with

install.do_egg_install()

should solve your problem. At least it works for me, but I would also appreciate a more detailed answer. Thanks!

Answer 2

According to https://stackoverflow.com/a/20196065 a more correct way to do this may be to override bdist_egg command.

You could try:

from setuptools.command.bdist_egg import bdist_egg as _bdist_egg

class bdist_egg(_bdist_egg):
    def run(self):
        call(["pip install -r requirements.txt --no-clean"], shell=True)
        _bdist_egg.run(self)

...

setup(...
    cmdclass={'bdist_egg': bdist_egg},  # override bdist_egg
)

It worked for me and install_requireis no more ignored. Nevertheless, I still don't understand why most people seem to override cmdclass install and do not complain about install_require being ignored.

Answer 3

I know this is an old question, but I ran into a similar problem. The solution I have found fixes this problem for me is very subtle: The install class you're setting in cmd_class must physically be named install. See this answer on a related issue.

Note that I use the class name install for my derived class because that is what python setup.py --help-commands will use.

You also should use self.execute(_func_name, (), msg="msg") in your post_install instead of calling the function directly

So implementing something like this should cause you to avoid the do_egg_install workaround implemented above by KEgg.

from setuptools.command.install import install as _install
...
def _post_install():
    #code here
class install(_install):
    def run(self):
        _install.run(self)
        self.execute(_post_install, (), msg="message here")