IPython Modules

Question

I have a few IPython scripts which have redundant functionality. I would like to refactor the common functionality into one module and include that modules in the existing script. The problem is it cannot be made a python module as the code uses Ipython's language extensions (!, $ etc). Is it possible to make a module having IPython code and include it in another IPython scripts?

Answer 1

You should not be saving the IPython extension stuff (?, !, %run) in files. Ever. Those are interactive tools and they are something you type with your hands but never save to a file.

1. Find the common features among your files. You have exactly four kinds of things that are candidates for this.

   • Imports (import)

   • Function definitions (def)

   • Class definitions (class)

   • Global variable assignments

   You must remove all IPython interactive features from this code. All of it.

2. Rewrite your scripts so they (1) import your common stuff, (2) do the useful work they're supposed to do.

   You must remove all IPython interactive features from this code. All of it.

   Now you can run your scripts and they're do their work like proper Python scripts are supposed.

You can still use IPython extension features like !, ? and %run when you're typing, but you should not save these into files.

Answer 2

technically if you save a script with the .ipy extension, ipython will see that and use all it's fancy stuff rather than passing directly to the python interpreter. however, i would generally recommend against this, and go the route of S.Lott above.

Answer 3

A lot of people strongly believe you're not supposed to have scripts with IPython syntax in them, but if you were curious enough (as I am) and are looking for some fun ways to mix python and shell scripts, you should checkout out my wrapper program on github

Example use case:

$ cat > example.ipy

rehashx

a = !ls -l | tail -n 3
print a.fields(0)

b = 'foo'
echo bar ${b}

$ ipyscript.py example.ipy

['-rw-r--r--', 'drwxr-xr-x', 'drwxrwxr-x']
bar foo

As it turns out, the IPython core also supports a (barely functional) version of the above script:

In [2]: IPython.core.interactiveshell.InteractiveShell.safe_execfile_ipy?
Type:       instancemethod
Base Class: <type 'instancemethod'>
String Form:<unbound method InteractiveShell.safe_execfile_ipy>
Namespace:  Interactive
File:       /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/core/interactiveshell.py
Definition: IPython.core.interactiveshell.InteractiveShell.safe_execfile_ipy(self, fname)
Docstring:
Like safe_execfile, but for .ipy files with IPython syntax.

Parameters
----------
fname : str
    The name of the file to execute.  The filename must have a
    .ipy extension.

Answer 4

If you enter the commands into an interactive version of IPython and then use the hist command (with -n to remove line numbers), IPython spits out all of the commands that you ran, with the actual python code used in place of !cd !ls, etc. Here's an example.

_ip.system("ls")
_ip.system("ls -F ")
_ip.magic("cd ")

http://ipython.scipy.org/moin/IpythonExtensionApi explains this object. This is basically what you need to do (adapted from the link):

import IPython.ipapi
_ip = IPython.ipapi.get()

Now all of the code you pasted from the IPython shell's hist command should work fine.

Answer 5

Have you had a look at the IPython module (pydoc IPython)? maybe you can access IPython's utilities through pure Python code.

Answer 6

You can do it.

Here is an example.

This is content of a.ipy file:

%run b.ipy

print(myvar)
print(myfunc())

This is content of b.ipy file:

myvar = !echo 1

def myfunc():
  tmp_var = !echo 2
  return tmp_var

As you can see b.ipy uses ! operator. When you execute a.ipy you get the following result:

ipython a.ipy 
['1']
['2']

So, you "import" "modules" not like you do it in python, but like you do it in shell with source.

But I'm not sure if it's right way, may be it is. At least it works and allows you to extract common functionality and reuse it from other script files.