How to Minimize mean square error using Python

Question

I want to minimise mean square error function to find best alpha value (decay rate) for my model.

Here is the description of my model:

time:              1st_month   2nd_month   3rd_month  4th_month  5th_month
Product_shipped     500            600         453      200        789

 If there is delay in products installed after shipping , we multiply by alpha

 Suppose alpha=-0.01
 We create a lower traingular matrix

 month   p1             p2           p3         p4            p5
  1    495.0249169              
  2     490.0993367 588.119204          
  3     485.2227668 582.2673201 439.6118267     
  4     480.3947196 576.4736635 435.2376159 192.1578878 
  5     475.6147123 570.7376547 430.9069293 190.2458849 750.5200159

M(1,1) is calculated as 500*(e^-alpha*month(=1))

M(2,1) is calculated as 500*(e^-alpha*month(=2))

M(2,2) is calculated as 600*(e^-alpha*month(=2))

So forth and so on.

Then Predicted Product shipment is sum across row:

Predicted_Installation
  495.0249169
  1078.218541
  1507.101914
 1684.263887
  2418.025197

We have originall Installation:

Original_Installation
   565
   1200
   1677
   1876
   2500

I want to minimise F(sum(Original_Installation-Predicted_Installation)^2) to find alpha which minimise this. How can we frame this or solve this in Python.

Answer 1

For this kind of problem, I would definitely start with scipy.otpimize methods.

I reproduce here an example on how to use it in your context:

import numpy as np
from scipy.optimize import minimize

ALPHA_TRUE = 0.5 # used only to generate some test data

def model(params, X):
    # here you need to implement your real model
    # for Predicted_Installation
    alpha = params[0]
    y_pred = np.exp(-alpha * X)
    return y_pred

def sum_of_squares(params, X, Y):
    y_pred = model(params, X)
    obj = np.sqrt(((y_pred - Y) ** 2).sum())
    return obj


# generate some test data
X = np.random.random(10) # this is "month" if I understood your problem correctly
Y = model([ALPHA_TRUE], X) # Original_Installation


# perform fit to find optimal parameters
# initial value for alpha (guess)
alpha_0 = 0.1

res = minimize(sum_of_squares, [alpha_0, ], args=(X, Y), tol=1e-3, method="Powell")
print(res)

The result looks like this:

   direc: array([[-1.12550246e-12]])
     fun: 0.0
 message: 'Optimization terminated successfully.'
    nfev: 225
     nit: 9
  status: 0
 success: True
       x: array(0.5)

You have to take a deep look at the documentation to find the best fitting method depending on whether alpha is bounded or not or whether you have constraints on your parameters.

I have also played around recently with the same kind of stuff using tensorflow gradient descent optimization (example: https://stellasia.github.io/blog/2020-02-29-custom-model-fitting-using-tensorflow/)