What is the difference between a Op and a Function

Question

Theano has Ops and functions.
What is the difference?

Functions seem nice and easy to define, eg:

x = T.dmatrix('x')
linmax = function([x], T.maximum(x,0))

Ops seem complex to define. All abstract classes and such but things like theano.tesnor.tanh and theano.tensor.nnet.sigmoid are defined as Ops. I'm not to sure on the difference.

How would I write the above linmax function as a Op?

Answer 1

theano.function() return a python object that is callable. So you can use it do the the computation you described when it was called.

Theano Ops are part of the symbolic graph that describe the computation that you want. Do not forget that Theano have two step as many other language as C and others. You first need to describe the computation that you want, then compile. In C, you define that computation in text file. In Theano, you describe it with a Theano symbolic graph and that graph include Ops.

Then you compile, with possible gcc for C and with theano.function() in Theano.

So Op is the element op the symbolic graph. It describe the computation done at one point in the graph. This page in Theano tutorial describe the graph in more detail:

http://deeplearning.net/software/theano/tutorial/symbolic_graphs.html#theano-graphs

This page describe how to make an Op in Theano:

http://deeplearning.net/software/theano/tutorial/extending_theano.html

You can skip the section for optional part. So you can skip most of that page if you don't plan to make one and just want to understand the usage.