How to apply CMA-ES optimization to an arbitrary user defined objective function in Python?

Question

I am new in using Python. These days I am trying to learn new optimization algorithms and python.

CMA-ES optimization algorithm source code in Python can be found here: CMA-ES.py

I have had all the necessary Python packages installed (numpy,matplotlib, winpython, and so on). It is also easy to run the testing functions provided by the source code, e.g.,

            >>> import cma
            >>> res = cma.fmin(cma.fcts.rosen, 4*[-1],1, ftarget=1e-6, restarts=3, verb_time=0, verb_disp=500, seed=3)

The desired customized objective function is from the nonlinear least square fittiing of data:

Data sets: 23x3

x        y      z
----------------------
1100.21 57.66   1.8
1157.88 57.79   1.7
1272.85 58.03   1.67
1330.34 58.22   1.67
1389.   57.69   1.7
1590.   57.01   1.67
1820.   55.42   1.6
2049.   59.35   1.5
2308.   58.32   1.56
2596.   57.28   1.6
2711.   57.13   1.368
2826.   55.61   1.33
2883.   54.79   1.315
2940.   53.78   1.325
3001.   54.41   1.3
3117.   55.93   1.2495
3291.   57.15   1.28
3377.   58.05   1.25
3522.   58.41   1.31
3725.   57.61   1.31
3899.   53.55   1.195
4015.   51.22   1.178
4188.   50.89   1.185

nonlinear model : a(1)--a(5) are parameters:

 z = a(1)*y^a(2)*x^a(3)+a(4)*x^a(5)

Answer 1

You could try

data = """
1100.21 57.66   1.8
1157.88 57.79   1.7
1272.85 58.03   1.67
1330.34 58.22   1.67
1389.   57.69   1.7
1590.   57.01   1.67
1820.   55.42   1.6
2049.   59.35   1.5
2308.   58.32   1.56
2596.   57.28   1.6
2711.   57.13   1.368
2826.   55.61   1.33
2883.   54.79   1.315
2940.   53.78   1.325
3001.   54.41   1.3
3117.   55.93   1.2495
3291.   57.15   1.28
3377.   58.05   1.25
3522.   58.41   1.31
3725.   57.61   1.31
3899.   53.55   1.195
4015.   51.22   1.178
4188.   50.89   1.185"""
data = np.array([line.split() for line in data.strip().split('\n')], dtype='f8')
x, y, z = data[:, 0], data[:, 1], data[:, 2]

def obj(a):
    z_hat = a[0]*y**a[1]*x**a[2]+a[3]*x**a[4]
    return ((z-z_hat)**2).sum()

import scipy.optimize as opt
print opt.minimize(obj, np.ones(5))

Or modify to use your solver. However, the function is quite scary and has lots of parameters.