How to debug a python module that needs to be executed with -m?

Question

Every debugger I tried out there expects a source file to debug. However Python does not always work this way.

I have a module that is a folder with __init__.py and __main__.py files inside, among others, and I usually execute that this way:

$ cd /parent/folder
$ python3 -m module_folder --help

If I don't use -m, relative imports fail. If I just pass the folder to pudb, pdb and others, debugger fails:

$ cd /parent/folder
$ python3 -m pdb module_folder
Traceback (most recent call last):
  File "/usr/lib64/python3.3/pdb.py", line 1658, in main
    pdb._runscript(mainpyfile)
  File "/usr/lib64/python3.3/pdb.py", line 1536, in _runscript
    with open(filename, "rb") as fp:
IsADirectoryError: [Errno 21] Is a directory: 'module_folder'
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /usr/lib64/python3.3/pdb.py(1536)_runscript()
-> with open(filename, "rb") as fp:
(Pdb)

How can I debug this? Preferrably with pudb, and without removing the relative imports.

Answer 1

Put this at the top of your __main__.py:

#!/usr/bin/env python3

# Declare itself as package if needed
if __name__ == '__main__' and __package__ is None:
    import os, sys, importlib
    parent_dir = os.path.abspath(os.path.dirname(__file__))
    sys.path.append(os.path.dirname(parent_dir))
    __package__ = os.path.basename(parent_dir)
    importlib.import_module(__package__)

# Continue with your code
do_things()

This way, these 2 commands become equivalent:

1. cd /parent/folder; python -m module_folder
2. python /parent/folder/module_folder/__main__.py

Just use the 2nd syntax for debugging:

pudb /parent/folder/module_folder/__main__.py

or

python3 -m pdb /parent/folder/module_folder/__main__.py

The same works for Python 2.

You can safely remove from the above code the part __name__ == '__main__' and because, as you are writing in __main__.py, it would always be True. However, it's a common practice to put it. See PEP 366 and this other answer.

Answer 2

I think you want to do this:

$ cd /parent/folder
$ python3 -m pdb -m module_folder

As it was, you were missing the second -m which was making Python think that module_folder was meant to be the name of a script to run.