scipy 'Minimize the sum of squares of a set of equations'

Question

I face a problem in scipy 'leastsq' optimisation routine, if i execute the following program it says

    raise errors[info][1], errors[info][0]
TypeError: Improper input parameters.

and sometimes index out of range for an array...

from scipy import *
import numpy
from scipy import optimize
from numpy import asarray
from math import *

def func(apar):
  apar = numpy.asarray(apar)
  x = apar[0]
  y = apar[1]
  eqn = abs(x-y)
  return eqn

Init = numpy.asarray([20.0, 10.0])
x = optimize.leastsq(func, Init, full_output=0, col_deriv=0, factor=100, diag=None, warning=True)
print 'optimized parameters: ',x
print '******* The End ******'

I don't know what is the problem with my func optimize.leastsq() call, please help me

Answer 1

leastsq works with vectors so the residual function, func, needs to return a vector of length at least two. So if you replace return eqn with return [eqn, 0.], your example will work. Running it gives:

optimized parameters:  (array([10., 10.]), 2)

which is one of the many correct answers for the minimum of the absolute difference.

If you want to minimize a scalar function, fmin is the way to go, optimize.fmin(func, Init).

The issue here is that these two functions, although they look the same for a scalars are aimed at different goals. leastsq finds the least squared error, generally from a set of idealized curves, and is just one way of doing a "best fit". On the other hand fmin finds the minimum value of a scalar function.

Obviously yours is a toy example, for which neither of these really makes sense, so which way you go will depend on what your final goal is.

Answer 2

Since you want to minimize a simple scalar function (func() returns a single value, not a list of values), scipy.optimize.leastsq() should be replaced by a call to one of the fmin functions (with the appropriate arguments):

x = optimize.fmin(func, Init)

correctly works!

In fact, leastsq() minimizes the sum of squares of a list of values. It does not appear to work on a (list containing a) single value, as in your example (even though it could, in theory).

Answer 3

Just looking at the least squares docs, it might be that your function func is defined incorrectly. You're assuming that you always receive an array of at least length 2, but the optimize function is insanely vague about the length of the array you will receive. You might try writing to screen whatever apar is, to see what you're actually getting.

If you're using something like ipython or the python shell, you ought to be getting stack traces that show you exactly which line the error is occurring on, so start there. If you can't figure it out from there, posting the stack trace would probably help us.