synthesized submodule: from A import B (ok) vs. import A.B (error)?

Question

I have a module modA, which contains a synthesized submodule modB (created with PyModule_New); now importing the module:

1. from modA import modB it is OK
2. import modA.modB fails.

What am I missing?

• modA.cpp (using boost::python, but it would be very likely the same with pure c-API of python):

  #include<boost/python.hpp>
  namespace py=boost::python;
  
  BOOST_PYTHON_MODULE(modA){
     py::object modB=py::object(py::handle<>(PyModule_New("modB")));
     modB.attr("__file__")="<synthetic>";
     py::scope().attr("modB")=modB;
  };
  

• compile with (g++ instead of clang++ works the same)

  clang++ -o modA.so modA.cpp -fPIC -shared  -lboost_python `pkg-config python --cflags --libs`
  

• test.py:

  import sys
  sys.path.append('.')
  from modA import modB
  import modA.modB
  

• python test.py (note the first import is just fine, it is the second one which fails):

  Traceback (most recent call last):
    File "test.py", line 4, in <module>
      import modA.modB
  ImportError: No module named modB
  

Answer 1

Thanks to this answer, I found the solution, which consists in sys.modules['modA.modB']=modB, but written in c++ in the module initialization function:

#include<boost/python.hpp>
namespace py=boost::python;

BOOST_PYTHON_MODULE(modA){
   py::object modB=py::object(py::handle<>(PyModule_New("modA.modB")));
   modB.attr("__file__")="<synthetic>";
   py::scope().attr("modB")=modB;

   // this was the missing piece: sys.modules['modA.modB']=modB
   py::extract<py::dict>(py::getattr(py::import("sys"),"modules"))()["modA.modB"]=modB;
};

Answer 2

Here is the pattern I use, hopefully it improves previous answers...

module.h:

...
#define STR(str) #str
#define MAKE_SUBMODULE(mod, submod) object submod##_module(handle<>(borrowed(PyImport_AddModule(STR(mod.submod)))));\
scope().attr(STR(submod)) = submod##_module;\
scope submod##_scope = submod##_module;\
...
export_submodule();
...

module.cpp:

BOOST_PYTHON_MODULE(modulename)
{
    export_submodule();
}

submodule.cpp:

export_submodule()
{
    // create submodule
    MAKE_SUBMODULE(modulename, submodulename)

    // all code below is now in submodule's scope
    ...
}

The "generated" code looks like this:

object submodulename_module(handle<>(borrowed(PyImport_AddModule("modulename.submodulename"))));
scope().attr("submodulename") = submodulename_module;
scope submodulename_scope = submodulename_module;

It looks similar to eudoxos answer, but the differencs seems to be in detail: Instead of PyImportNew() I use PyImportAdd(). Therefor I don't need the last line to get the from module.submodule import * statement working.

Answer 3

modB is not a file (i.e. a module), but some object in modA, hence it cannot be imported.

You may want to see Python's modules docs.