'python setup.py nosetests' reports that package's __init__.py is not covered by tests

Question

When I test https://bitbucket.org/petar/beam_integrals with python setup.py nosetests I get a 96% coverage:

----------------------------------------------------------------------
XML: nosetests.xml
Name                                             Stmts   Miss  Cover   Missing
------------------------------------------------------------------------------
beam_integrals                                       7      7     0%   1-24
beam_integrals.beam_types                           79      0   100%
beam_integrals.characteristic_equation_solvers      65      0   100%
beam_integrals.exceptions                            6      0   100%
beam_integrals.utils                                14      0   100%
------------------------------------------------------------------------------
TOTAL                                              171      7    96%
----------------------------------------------------------------------
Ran 634 tests in 178.245s

OK (SKIP=3)

However if I test the same code with nosetests I get full coverage:

----------------------------------------------------------------------
XML: nosetests.xml
Name                                             Stmts   Miss  Cover   Missing
------------------------------------------------------------------------------
beam_integrals                                       7      0   100%
beam_integrals.beam_types                           79      0   100%
beam_integrals.characteristic_equation_solvers      65      0   100%
beam_integrals.exceptions                            6      0   100%
beam_integrals.utils                                14      0   100%
------------------------------------------------------------------------------
TOTAL                                              171      0   100%
----------------------------------------------------------------------
Ran 634 tests in 179.226s

OK (SKIP=3)

It seems python setup.py nosetests isn't properly reporting test coverage of the beam_integrals module.

This issue has been verified on:

1. our continuous integration server running Ubuntu 10.04 Server 32bit

2. a freshly created Rackspace Cloud Server running Ubuntu 12.04 Server 64bit with the following setup:

   $ sudo aptitude update
   $ sudo aptitude upgrade
   $ sudo reboot
   
   $ sudo aptitude install python-pip mercurial git-core python-gmpy python-dev
   $ hg clone https://bitbucket.org/petar/beam_integrals
   $ sudo pip install -r beam_integrals/requirements.txt
   $ sudo pip install -r beam_integrals/requirements-test.txt
   
   $ cd beam_integrals
   $ python setup.py nosetests
   $ nosetests
   

Answer 1

python setup.py nosetests is importing beam_integrals before starting coverage testing, so it's already been imported when coverage measurement is happening. This is because your setup.py imports beam_integrals directly. This isn't a bad thing to do, lots of Python projects do this to get version information from the code itself rather than duplicating it in the setup.py.

When you use nosetests, it knows to begin the coverage measurement before importing anything. Actually, it probably imports things, unimports them, starts coverage, then runs the code, which imports things again.

I'm not sure what you can do about this, other than to use nosetests to run your tests.

Answer 2

To complete Ned answer, I know two solutions to circumvent the problem without duplicating the version in your files.

1. Remove the modules from sys.modules after importing your module

   import beam_integrals
   for mod in [m for m in sys.modules.keys() if m.startswith('beam_integrals')]:
       del sys.modules[mod]
   

   Coverage report will now include all the files.
   However, in your case it would still require to have sympy installed.
   That's the reason I moved to the next solution.

2. Read module version without importing it:

   def get_version():
       with open('beam_integrals/__init__.py') as f:
           for line in f:
               if line.startswith('__version__'):
                   return eval(line.split('=')[-1])
   
   setup(
       name='beam_integrals',
       version=get_version(),
   

   I saw this used here: https://github.com/PyCQA/pycodestyle/blob/2.3.1/setup.py