Running an IPython/Jupyter notebook non-interactively

Question

Does anyone know if it is possible to run an IPython/Jupyter notebook non-interactively from the command line and have the resulting .ipynb file saved with the results of the run. If it isn't already possible, how hard would it be to implement with phantomJS, something to turn the kernel on and off, and something to turn the web server on and off?

To be more specific, let's assume I already have a notebook original.ipynb and I want to rerun all cells in that notebook and save the results in a new notebook new.ipynb, but do this with one single command on the command line without requiring interaction either in the browser or to close the kernel or web server, and assuming no kernel or web server is already running.

example command:

$ ipython notebook run original.ipynb --output=new.ipynb

Answer 1

Yes it is possible, and easy, it will (mostly) be in IPython core for 2.0, I would suggest looking at those examples for now.

[edit]

$ jupyter nbconvert --to notebook --execute original.ipynb --output=new.ipynb

It is now in Jupyter NbConvert. NbConvert comes with a bunch of Preprocessors that are disabled by default, two of them (ClearOutputPreprocessor and ExecutePreprocessor) are of interest. You can either enabled them in your (local|global) config file(s) via c.<PreprocessorName>.enabled=True (Uppercase that's python), or on the command line with --ExecutePreprocessor.enabled=True keep the rest of the command as usual.

The --ExecutePreprocessor.enabled=True has convenient --execute alias that can be used on recent version of NbConvert. It can be combine with --inplace if desired

For example, convert to html after running the notebook headless :

$ jupyter nbconvert --to=html --execute RunMe.ipynb

converting to PDF after stripping outputs

$ ipython nbconvert --to=pdf --ClearOutputPreprocessor.enabled=True RunMe.ipynb

This (of course) does work with non-python kernels by spawning a <insert-your-language-here> kernel, if you set --profile=<your fav profile>. The conversion can be really long as it needs to rerun the notebook. You can do notebook to notebook conversion with the --to=notebook option.

There are various other options (timeout, allow errors, ...) that might need to be set/unset depending on use case. See documentation and of course jupyter nbconvert --help, --help-all, or nbconvert online documentation for more information.

Answer 2

Until this functionality becomes part of the core, I put together a little command-line app that does just what you want. It's called runipy and you can install it with pip install runipy. The source and readme are on github.

Answer 3

Run and replace original .ipunb file:

jupyter nbconvert --ExecutePreprocessor.timeout=-1 --to notebook --inplace --execute original.ipynb

Answer 4

To cover some features such as parallel workers, input parameters, e-mail sending or S3 input/output... you can install jupyter-runner

pip install jupyter-runner

Readme on github: https://github.com/omar-masmoudi/jupyter-runner

Answer 5

One more way is to use papermill, it has Command Line Interface

Usage example: (you need to specify output path for execution results to be stored)

papermill your_notebook.ipynb logs/yourlog.out.ipynb

You also can specify required params if you wish with -p flag for each param:

papermill your_notebook.ipynb logs/yourlog.out.ipynb -p env "prod" -p tests "e2e"

one more related to papermill reply - https://stackoverflow.com/a/55458141/2957102

Answer 6

You can just run the iPython-Notebook-server via command line:

ipython notebook --pylab inline

This will start the server in non-interactive mode and all output is printed below the code. You can then save the .ipynb-File which includes Code & Output.