how to know if my Python tests are running in coverage mode?

Question

I am running Ned Batchelder's coverage module on continuous integration using Travis CI but I want to run only integration tests and skip functional ones because they take too long and coverage measurement is not affected by them.

I created a special configuration for this, but I want to know if there is an alternate method of knowing, inside a Python script, is the code is being run by coverage or not.

Answer 1

Travis CI provides a couple of environment variables that can be used for this; in my case any of this will serve:

CI=true
TRAVIS=true

even as both answers provided before were really useful, I think this solution is easier to implement for what I need.

Answer 2

nose can definitely help with it:

• Cover: code coverage plugin
• Attribute selector plugin
  • you can mark tests with @attr("no-coverage") decorator and run your coverage tests with -a '!no-coverage' option
• nose-exclude plugin
  • you can exclude specific test dirs and test files from running using --exclude-dir and --exclude-dir-file options

Hope that helps.

Answer 3

Based on the wording of your question I am assuming that you are not limiting what tests you are running with coverage and would like the functional tests to notice they are being run with coverage, and do nothing. A hacky way might be to look at sys.argv in the functional tests and do things differently if you detect coverage usage. But I think a better approach would be to have functional tests and unit tests in separate sibling directories, and tell coverage to run only the tests in the unit test directory. Potentially you could also use the --omit option to limit which tests are being run.

Answer 4

I needed to determine if my tests were running under plain debug mode, with coverage, or just normally. After a good deal of experimentation I came up with this:

import sys

# Detect PyCharm debugging mode
get_trace = getattr(sys, 'gettrace', lambda: None)

if get_trace() is None:
    debug = False
    print('runnin normsies')
else:
    debug = True
    print('debuggin')
    if 'coverage' in sys.modules.keys():
        print('covered')

Not sure how robust it is, but it works for me.

Answer 5

Here's an implementation of the check whether a test is run in coverage mode. The nice thing about this is that you can use gettrace_result to check other conditions, e.g., whether the test is run by a debugger instead of coverage:

import sys

def is_run_with_coverage():
    """Check whether test is run with coverage."""
    gettrace = getattr(sys, "gettrace", None)

    if gettrace is None:
        return False
    else:
        gettrace_result = gettrace()

    try:
        from coverage.pytracer import PyTracer
        from coverage.tracer import CTracer

        if isinstance(gettrace_result, (CTracer, PyTracer)):
            return True
    except ImportError:
        pass

    return False

You can use pytest.mark.skipif to skip tests that shouldn't be run in coverage mode.

@pytest.mark.skipif(is_run_with_coverage())
def test_to_skip_in_coverage_mode():
    ...

Answer 6

From the coverage.py documentation:

Coverage.py sets an environment variable, COVERAGE_RUN to indicate that your code is running under coverage measurement.

Important: this option is only available in version 6.1 and later of the coverage module.

If detectcoverage.py contains:

import os
def detect_coverage():
    return os.environ.get('COVERAGE_RUN', None) is not None

if detect_coverage():
    print("running in coverage mode")
else:
    print("not running in coverage mode")  

then running that looks like:

$ coverage run detectcoverage.py
running in coverage mode

$ python detectcoverage.py 
not running in coverage mode