Defining a function with a loop in Theano

Question

I want to define the following function of two variables in Theano and compute its Jacobian:

f(x1,x2) = sum((2 + 2k - exp(k*x1) - exp(k*x2))^2, k = 1..10)

How do I make a Theano function for the above expression - and eventually minimize it using its Jacobian?

Answer 1

Since your function is scalar, the Jacobian reduces to the gradient. Assuming your two variables x1, x2 are scalar (looks like it from the formula, easily generalizable to other objects), you can write

import theano
import theano.tensor as T

x1 = T.fscalar('x1')
x2 = T.fscalar('x2')

k = T.arange(1, 10)

expr = ((2 + 2 * k - T.exp(x1 * k) - T.exp(x2 * k)) ** 2).sum()

func = theano.function([x1, x2], expr)

You can call func on two scalars

In [1]: func(0.25,0.25)
Out[1]: array(126.5205307006836, dtype=float32)

The gradient (Jacobian) is then

grad_expr = T.grad(cost=expr, wrt=[x1, x2])

And you can use updates in theano.function in the standard way (see theano tutorials) to make your gradient descent, setting x1, x2 as shared variables in givens, by hand on the python level, or using scan as indicated by others.