Getting Attempted relative import in non-package error in spite of having __init__.py

Question

I have a package cclogger. This directory has a __init__.py file with some code to load the configuration. When I try to run the file api_main.py in that directory using the following command...

python -m cclogger.api_main

I get the following erro:-

config loaded
Instantiating DB with: cclogger/test123@localhost:x
Instantiated ParseCentral
Register parser called by : CitiIndia
Registered parser for email:  CitiAlert.India@citicorp.com
Instantiated SmsParseCentral
Register parser called by : Citi Bank
Registered sms parser for address:  lm-citibk
Register parser called by : HDFC Bank
Registered sms parser for address:  am-hdfcbk
Traceback (most recent call last):
  File "/Users/applegrew/Dropbox/Credit Expense/cclogger/cclogger/api_main.py", line 4, in <module>
    from .bottle import run, default_app, debug, get
ValueError: Attempted relative import in non-package

The messages displayed above the error are from modules in the same package which were imported by __init__.py.

The code in api_main.py is:-

import re
import os

from .bottle import run, default_app, debug, get
from .common_util import date_str_to_datetime, UTCOffset, date_filter

#app = Bottle()

default_app().router.add_filter('date', date_filter)

from . import api, dev

@get('/index')
def index():
    return "CCLogger API main live and kicking."

if dev:
    debug(True)
    run(reloader=True, port=9000)
else:
    os.chdir(os.path.dirname(__file__))
    application = default_app()

I have python 2.7.1.

What am I doing wrong? You can see the full code at https://github.com/applegrew/cclogger/tree/master/cclogger

Answer 1

You cannot run a python module directly as a script (I don't really know the reason why).

EDIT : The reason is explained in the PEP338 which is the spec for the "-m" option.

The release of 2.5b1 showed a surprising (although obvious in retrospect) interaction between this PEP and PEP 328 - explicit relative imports don't work from a main module. This is due to the fact that relative imports rely on __name__ to determine the current module's position in the package hierarchy. In a main module, the value of __name__ is always __main__ , so explicit relative imports will always fail (as they only work for a module inside a package).

For the 2.5 release, the recommendation is to always use absolute imports in any module that is intended to be used as a main module

To test your application, encapsulate api_main in a function and create a top-level main.py file which will run the main loop :

cclogger/api_main.py :

import re
import os


from .bottle import run, default_app, debug, get
from .common_util import date_str_to_datetime, UTCOffset, date_filter

#app = Bottle()


def main():
    default_app().router.add_filter('date', date_filter)

    from . import api, dev

    @get('/index')
    def index():
        return "CCLogger API main live and kicking."

    if dev:
        debug(True)
        run(reloader=True, port=9000)
    else:
        os.chdir(os.path.dirname(__file__))
        application = default_app()

And /main.py :

from cclogger import api_main


if __name__ == '__main__':
    api_main.main()

You can run your application by typing python main.py, python -m main or python -c "import cclogger.api_main; api_main.main()" .

PS : thanks for linking the complete source, it's always much more helpful than the stubs provided with the question.

Answer 2

Relative imports work fine in modules executed using python -m since Python 2.6. As explained in http://www.python.org/dev/peps/pep-0366, the __package__ attribute is automatically set on the module when executed this way so that relative imports work. The piece that's causing the failure here is the reloader, which apparently fails to set __package__.

Edit: Glancing at the code, I see that this is a bottle API. I didn't check bottle, but it likely has the same problem as Flask's reloader, which attempts the restart by trusting what's in sys.argv. The problem is that the code that handles python -m (which is in the runpy module) modifies sys.argv, removing the "-m" and translating the module name to a filename (which is very frustrating, but I'm sure they had good reason). Thus, what the reloader actually executes has no "-m" and the bit that sets __package__ never fires, resulting in no support for relative imports.