Accessing shared modules from child packages

Question

I'm having difficulties with using internal imports inside my projects. This is a partial tree structure of my project:

app
  |- Gui.py
  |- Main.py
  |- logger.py
  |- config.py
  |- WebParser (package)
        |- __init__.py
        |- LinksGrabber.py
        |- LyricsGrabber.py
        |- ImagesGrabber.py
  |- (Many other packages...)

the logger.py and config.py modules are required in every package's modules, and are independent (uses only bulit-in modules). It is tricky to access these modules from inside packages.

This is how I tried to achieve it, for enabling configuration access and logging feature in WebParser\LinksGrabber.py:

# WebParser\__init__.py:
sys.path.append('..') # for outside-package modules
import config
from logger import log

# WebParser\LinksGrabber.py:
import WebParser
config = WebParser.config
log = WebParser.log

The Problems:

• This has code smell. I bet there is a better way to achieve this behaviour.
• I want to call import WebParser and use WebParser.LinksGrabber and WebParser.LyricsGrabber right away, without implicitly importing them. This can be done with importing the modules inside __init__.py, but it isn't possible because every package's module imports the package itself, and it will issue recursive imports.

Can you suggest a better implemention, or a different code design?

Answer 1

You should make app a package by giving it an __init__.py file. The python relative import system only works inside packages. Then inside your WebParser modules, you can do from .. import config, from .. import Gui, etc.

As for importing WebPackage from the packages inside it, that is a bit of a code smell. Why do you need to do that? Using relative imports you could instead, for instance, have from . import LinksGrabber inside ImagesGrabber, etc., to access what you need. If there are functions that are part of the WebParser package that are needed by many of the submodules, you should pull those out into a separate module in WebParser.

Answer 2

I would make the entire outer (app) directory a python package (with an __ init__.py).

app_files
  |- ***setup.py***
  |- app
    |- ***__init__.py***
    |- Gui.py
    |- Main.py
    |- logger.py
    |- config.py
    |- WebParser (package)
          |- __init__.py
          |- LinksGrabber.py
          |- LyricsGrabber.py
          |- ImagesGrabber.py
    |- (Many other packages...)

setup.py would be something simple like this:

#!/usr/bin/env python

from distutils.core import setup

setup(name='app',
      version='1.0',
      description='My app',
      author='Greg Ward',
      packages=['app'],
     )

then you can run python setup.py install, to permanently install 'app' into the python path. This is, in my opinion, the best way to do it without resorting to sys.path hacks everywhere.

Then, from anywhere in python you can refer to any of your files from the full dotted paths

i.e.

import app.logger
import app.config
import app.WebParser

The only way to make LinksGrabber and LyricsGrabber available from just an import app.WebParser would be to import them from app.WebParser.__ init__.

Answer 3

It seems that you could use relative imports here:

from .. import config
from ..logger import log