How to find all child modules in Python?

Question

While it is fairly trivial in Python to import a "child" module into another module and list its attributes, it becomes slightly more difficult when you want to import all child modules.

I'm building a library of tools for an existing 3D application. Each tool has its own menu item and sub menus. I'd like the tool to be responsible for creating its own menus as many of them change based on context and templates. I'd like my base module to be able to find all child modules and check for a create_menu() function and call it if it finds it.

What is the easiest way to discover all child modules?

Answer 1

using dir() and imp module

Answer 2

I think the best way to do this sort of plugin thing is using entry_points and the API for querying them.

Answer 3

When I was a kind and just beginning programming in Python I've written this for my modular IRC bot:

    # Load plugins

    _plugins = []

    def ifName(name):
        try:
            return re.match('([^_.].+)\.[^.]+', a).group(1)
        except:
            return None

    def isValidPlugin(obj):
        from common.base import PluginBase
        try:
            if obj.__base__ == PluginBase:
                return True
            else:
                return False
        except:
            return False

    plugin_names = set(ifilter(lambda a: a!=None, [ifName(a) for a in os.listdir(os.path.join(os.getcwd(), 'plugins'))]))
    for plugin_name in plugin_names:
        try:
            plugin = __import__('plugins.'+plugin_name, fromlist=['plugins'])
            valid_plugins = filter(lambda a: isValidPlugin(a), [plugin.__getattribute__(a) for a in dir(plugin)])
            _plugins.extend(valid_plugins)
        except Exception, e:
            logger.exception('Error loading plugin %s', plugin_name)

    # Run plugins

    _plugins = [klass() for klass in _plugins]

It's not secure or "right" way, but maybe it we'll be useful nevertheless. It's very old code so please don't beat me.

Answer 4

The solution above traversing the filesystem for finding submodules is ok as long as you implement every plugin as a filesystem based module.

A more flexible way would be an explicit plugin list in your main module, and have every plugin (whether a module created by file, dynamically, or even instance of a class) adding itself to that list explicitly. Maybe via a registerPlugin function.

Remember: "explicit is better than implicit" is part of the zen of python.

Answer 5

You can try globbing the directory:

import os
import glob

modules = glob.glob(os.path.join('/some/path/to/modules', '*.py'))

Then you can try importing them:

checked_modules
for module in modules:
    try:
        __import__(module, globals(), locals()) # returns module object
    except ImportError:
        pass
    else:
        checked_modules.append(module)