Python Rope: How to Find all missing imports and errors in all sub modules refactoring

Question

I am trying to find all missing import statements and errors for each module and its sub modules.

Is there a dedicated tool for what I am trying to do?

The code that I wrote, but seems really terrible and maybe something like this exists already?:

import os
def find_missing_imports(walk):
    for items in walk:
        d = items[0]
        f_list = items[1]
        for f in f_list:
            module = f[:-3]
            # posix_path
            module_path = d.lstrip('.').replace('/','.').lstrip('.')
            try:
                __import__(module_path, fromlist=[module])
            except IndentationError, e:
                #print(f,e)
                pass
            except NameError, e:
                print(d,f,e)
                pass
            except Exception, e:
                print(f,e)
                pass

walk = [[root,files] for root,dirs,files in os.walk('.') for fn in files if  fn.endswith('.py')]
find_missing_imports(walk)

Outputs:

.[snip]
('./Sky_Group_Inventory_Scanner-wxpython/display_image/Dialogs', 'ImageSelectionFrame.py', NameError("name 'wx' is not defined",))
('./Sky_Group_Inventory_Scanner-wxpython/display_image/Dialogs', 'ItemSpecificsDialog.py', NameError("name 'wx' is not defined",))
('./Sky_Group_Inventory_Scanner-wxpython/display_image/Dialogs', 'ReturnCorrectWatchTitle.py', NameError("name 'wx' is not defined",))
.[snip]

My project before refactoring was a mess but sort of useful, now its broken after refactoring.

After reading 'The Pragmatic Programmer' based on suggestions from my initial post on codereview:

• https://codereview.stackexchange.com/questions/46223/wxpython-item-information-scraper

I have been digging around in the source code of:

/usr/local/lib/python2.7/dist-packages/rope

Documentation for ROPE seems a little sparse. I have also been using Ninja-IDE, but haven't been able to find a solution for the problem that I am facing.

Overall I think I missed the boat on what refactoring is all about.

The current parent directory layout can be seen here.

In comparison to what it was before.

Any help, on filling in missing terminology, or on what I am even asking would be great.

Solution:

pylint -E /path/to/module

Answer 1

pip install pylint

Point pylint to the folder/module in question:

pylint /path/to/module > pylint_output

Where /path/to/module is the location of the python module you're interested in looking at.

For example:

my_project
   |____  moduleA
             |____  __init__.py
             |____  A.py
   |____  moduleB
             |____  __init__.py
             |____  B.py

• ./my_project/moduleA
• ./my_project/moduleB

This will create a file with Global Evaluations on:

• undefined-variable <-- This is what I was looking for in each file.
• invalid-name
• line-too-long
• superfluous-parens
• bad-whitespace
• attribute-defined-outside-init
• missing-docstring
• bad-indentation
• bad-continuation
• broad-except
• unused-argument
• unused-import
• unused-variable
• no-self-use
• no-member
• fixme
• unnecessary-pass
• multiple-statements
• too-many-statements
• duplicate-code
• too-many-locals
• missing-final-newline
• too-many-instance-attributes
• too-many-branches
• redefined-builtin
• too-many-public-methods
• syntax-error
• relative-import
• maybe-no-member
• import-error
• super-on-old-class
• bare-except
• undefined-loop-variable
• too-many-return-statements
• too-many-arguments
• too-few-public-methods
• star-args
• reimported
• indexing-exception
• unreachable
• too-many-lines
• redefined-outer-name
• property-on-old-class
• pointless-string-statement
• pointless-statement
• old-style-class
• no-name-in-module
• global-variable-undefined
• expression-not-assigned
• bad-except-order
• assignment-from-none

Of interest and the direct answer to my question is that within the pylint results there will be lines that have this sort of layout:

************* Module module_name.sub_module.class_name.method_name
R: line_no, column: Issue description 'some_name' (issue-type)
C: line_no, column: Issue description 'some_name' (issue-type)
W: line_no, column: Issue description 'some_name' (issue-type)
E: line_no, column: Issue description 'some_name' (issue-type)
F: line_no, column: Issue description 'some_name' (issue-type)
************* Module module_name.sub_module.class_name.method_name
R: line_no, column: Issue description 'some_name' (issue-type)
C: line_no, column: Issue description 'some_name' (issue-type)
W: line_no, column: Issue description 'some_name' (issue-type)
E: line_no, column: Issue description 'some_name' (issue-type)
F: line_no, column: Issue description 'some_name' (issue-type)    

• [R]efactor for a “good practice” metric violation
• [C]onvention for coding standard violation
• [W]arning for stylistic problems, or minor programming issues
• [E]rror for important programming issues (i.e. most probably bug)
• [F]atal for errors which prevented further processing

So an issue-type of (undefined-variable) in most of my cases indicate modules that have not been imported. pylint -E /path/to/module will return only the undefined-variable errors.