import modules when starting scripts in command-line

Question

I'm running a python3 script/program that uses my private module MyModule.py. It is placed in my site-packages folder.

When running the script from within python (with exec(open("path\to\my\script.py").read()) ), everything works fine. Also works import MyModules.

However, when I call from within cmd python "path\to\my\script.py", I get the following error:

C:\Users\jochen.tackenberg>python H:\@work.Jochen\plot_rzsaldo.0.5.3.2_topng.py
Traceback (most recent call last):
  File "H:\@work.Jochen\plot_rzsaldo.0.5.3.2_topng.py", line 14, in <module>
    import rzsaldo_data_current
ImportError: bad magic number in 'rzsaldo_data_current': b'\x03\xf3\r\n'

It is exactly the same script as I load with my exec command. Even if I add manually my site-packages to my PYTHONPATH using

setlocal
set PYTHONPATH=C:\Python33\Lib\site-packages

it does not work.

After several requests, I'm pasting here some snippets of the code to show what I'm trying to do: (this is only the module, not the importing script...)

import datetime
import urllib
import urllib.error
# import pdb

def _today():
# returns todays date

    todays_date = datetime.datetime.now()
    return str(todays_date.day) + '.' + str(todays_date.month) + '.' + str(todays_date.year)


class _RegelzonenDataClass():
# This class constructs the data objects, which contains the read-out information from the URL

    def __init__(self):
        self.date = []
        self.time = []
        self.fulltime = []

    def initialize_produkt_container(self, produkt):
        if produkt == 'RZ_SALDO':
            self.rz_saldo = []
        else:
            self.neg_request = []
            self.pos_request = []



class SomeOnlineData(object):
# This class can read in and contain all data necessary 

    def __init__(self, von=None, bis=None, uenbId='Netzregelverbund', produkt='RZ_SALDO'):

        self.url = ''
        self._raw_data = []
        self.data = _RegelzonenDataClass()

    # retrieve data from some webpage and strip data
        self.get_data(von, bis, uenbId, produkt )
        self.read_data()

    def get_data(self, von, bis, uenbId, produkt ):

        if von is None:
            self.von = _today()
        else:
            self.von = von

        if bis is None:
            self.bis = _today()
        else:
            self.bis = bis


        self.url = 'some.url.com/index.php?here' + I_paste + '&some=' + 'argumemts'
        self._raw_data = urllib.request.urlopen( self.url )


    def read_data(self):
    # process the raw html response into the data class


        self.data.initialize_produkt_container(self.produkt)


        for raw_data_line in self._raw_data:

            # check whether the current line is part of the header; if so, skip
            dummy_line_skipper = raw_data_line[0:1]
            if not dummy_line_skipper.isdigit(): continue


            dummy_string = str(raw_data_line).split(';')

            self.data.date.append( datetime.datetime.strptime( dummy_string[0][2:], '%d.%m.%Y' ) )
            self.data.somemore.append( some_data )

            # the data is given in weird german standard - decimal seperator is ','
            self.data.data_column.append( float( dummy_string[2].translate(str.maketrans(',.','.,' ) ).replace(',','' ) ) )

What is puzzling me most is the fact that it does not complain at all if I import it from within python.

Any ideas? Thanks a lot!

Cheers Jochen

UPDATE: Since none of the suggestions work, for the moment I simply copied all of my module code to the main program. That is nasty, but works...

Answer 1

After researching a bit on this topic and also based on the comments to thw query, the most likely probability for this error to occur is the mixing up of Python 2.7 and Python 3 versions and the presence of some stale .pyc files within the site-packages directory.

When you import a module within another module, it works fine because, the same Python interpreter (v3.3) is used to load the Python modules and hence there is no problem encountered.

since you are facing a problem only when you run it from the command line, it would help to verify which version of python you are using to execute these scripts.

At the command prompt type :

python -version

this should give you the version of python you are currently using to run your script. Make sure that it is version 3.3. Else, you will need to modify the PATH environment variable to make the path for the 3.3 version of the interpreter ( say c:\python33 ) come before any other python interpretor version (say c:\python27 ). You can do this as below:

SET PATH=c:\python33;%PATH%

assuming Python 3.3 interpreter is present within c:\python33

An edit based on the response from JoChen

One of the following cases is definitely true:

• plot_rzsaldo.0.5.3.2_topng.py supports only the 2.7 version of Python. This is the file that is causing the Bad magic Number error; not your module file.
• There is a stray plot_rzsaldo.0.5.3.2_topng.pyc file somewhere within the directories from where python loads it's files which is generated by Python 2.7.

How did I say that the version of python is 2.7? - Well the answer lies in the magic number which is printed as \x03\xF3\r\n (L.S.Byte -03 and M.S.Byte- F3). This when converted to decimal gives 62211 which is the magic number for Python 2.7a0 62211 (refer this link for details). Another interesting link also details the structure of .pyc files. This link details about the sensitivity of .pyc files to the interpreter version.

Given that we do not have access to all the source code and the modules being imported, this is the best that I can answer after my research.

Answer 2

This is caused by the linker pyc, have a look on the directory of your scripts, you may have used two differents versions of python exp 2... 3... etc

just to rm -f the pyc file that causes the problem and you will fix it.