Purpose and usage of PyModule_New

Question

At face value, the C-API function PyModule_New and PyModule_NewObject obviously creates a new module object.

The official Python Documentation provides the following explanation for PyModule_NewObject:

Return a new module object with the name attribute set to name. Only the module’s doc and name attributes are filled in; the caller is responsible for providing a file attribute.

PyModule_New does the same thing, except it accepts a C string (char*) as an argument for the module name, instead of a PyObject* string.

Okay, so this is pretty straightforward, but...

My question is: what is the use of calling the API function PyModule_NewObject?

Sure, theoretically it would be great for a situation where you want to dynamically create a new module. But the problem is that, in practice, after creating a new module object, the only way to do anything useful to it would be to add objects (like methods, classes, variables, etc.) to the module's __dict__ attribute. This way users of the module could import it and actually do something with it.

The problem is that the __dict__ attribute of a module is read-only:

>>> import re
>>> x = re
>>> re.__dict__ = { "foo" : "bar" }
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: readonly attribute

So, in practice, there's really no way to do anything useful with a dynamically created module, as far as I can see. So what then, is the purpose of the C API function PyModule_New?

Answer 1

The PyModule_New is the constructor for module objects. It's exposed to pure Python code too, as the __new__ method of the types.ModuleType class.

It probably is pretty rare for user code to need to use either of those, since you usually get modules by importing them. However, the machinery used by the Python interpreter uses PyModule_New to make the module objects when an import is requested.

You can see this in import.c in the Python source:

/* Get the module object corresponding to a module name.
First check the modules dictionary if there's one there,
if not, create a new one and insert it in the modules dictionary.
Because the former action is most common, THIS DOES NOT RETURN A
'NEW' REFERENCE! */

PyImport_AddModule(const char *name)
{
    PyObject *modules = PyImport_GetModuleDict();
    PyObject *m;

    if ((m = PyDict_GetItemString(modules, name)) != NULL &&
        PyModule_Check(m))
        return m;
    m = PyModule_New(name);
    if (m == NULL)
        return NULL;
    if (PyDict_SetItemString(modules, name, m) != 0) {
        Py_DECREF(m);
        return NULL;
    }
    Py_DECREF(m); /* Yes, it still exists, in modules! */

    return m;
}

As for how to set values in a new module object, you can use regular attribute access. In Python code (rather than C), that's simple:

import types

mymodule = types.ModuleType("mymodule")
mymodule.foo = "foo"

Note that a module created this way can't be imported anywhere else unless you do some additional work. For instance, you can add it to the modules lookup dictionary, sys.modules:

import sys

sys.modules["mymodule"] = mymodule

Now other modules can import mymodule by name.