Python3 plugin system

Question

I'm trying to create a plugin framework similar to yapsy (unfortunately yapsy is not python3 compatible).

My code looks like this:

root
   main.py
   plugins/
       __init__.py
       PluginManager.py
       UI/
           __init__.py
           textui.py

In PluginManager.py I defined the following class:

class PluginMetaclass(type):
    def __init__(cls, name, base, attrs):
        if not hasattr(cls, 'registered'):
            cls.registered = []
        else:
            cls.registered.append((name,cls))

class UI_Plugins(object):
    __metaclass__ = PluginMetaclass

    #...some code here....

    def load():
         #...some code here too...

        if "__init__" in  os.path.basename(candidate_filepath):
            sys.path.append(plugin_info['path'])
        try:
            candidateMainFile = open(candidate_filepath+".py","r")  
            exec(candidateMainFile,candidate_globals)
        except Exception as e:
            logging.error("Unable to execute the code in plugin: %s" % candidate_filepath)
            logging.error("\t The following problem occured: %s %s " % (os.linesep, e))
            if "__init__" in  os.path.basename(candidate_filepath):
                sys.path.remove(plugin_info['path'])
            continue

where candidate_filepath contains the plugin path.

textui.py contains the following:

from root.plugins.PluginManager import UI_Plugins

class TextBackend(UI_Plugins):
    def run(self):
        print("c")

When I try to load the plugin I get this error:

No module named plugins.PluginManager 

How can I solve this problem?

Answer 1

The import statement

from root.plugins.PluginManager import UI_Plugins

doesn't work because root is not a package.

However, if the application is started with

python3 root/main.py

then root doesn't actually need to be a package.

All you need to do is change the the import statement in textui.py to

from plugins.PluginManager import UI_Plugins

and everthing should work correctly.

The reason this works, is because the directory of the currently running script is always automatically added to the start of sys.path. In your case, this will be the root, and since plugins is a package within that directory, it can be directly imported from anywhere within your application. So, as long as your main script remains where it is, there should be no need for any other path manipulations.

Answer 2

Sorry, this is certainly not a direct answer to your question but if you're trying to develop something very close to yapsy for python3, then you may be interested by the new version of yapsy in which I've released a couple of python3-compatible packages:

https://sourceforge.net/projects/yapsy/files/Yapsy-1.9/

(see Yapsy-1.9_python3-py3.2.egg or Yapsy-1.9-python3.tar.gz)

The source code being on a specific branch:

http://yapsy.hg.sourceforge.net/hgweb/yapsy/yapsy/file/91ea058181ee

Answer 3

1. In order to have a package, you need to have a __init__.py file in a directory. It may be empty, but it must be there, in both "root" and "plugin" directories.
2. The name of the directory is the name of the namespace, and therefore they must match carefully. In your case, you need to use from root.plugin.PluginManager import UI_Plugins
3. Finally, in order for import to work, the package must be in your PYTHONPATH, (see The Module Search Path, in the language documentation). You can do this either by adding the directory to the PYTHONPATH environment variable, either in code, by adding it to the sys.path list.