Multiple Linear Regression in C#

Question

I want to make a multiple linear regression in C#. I am trying to achieve this with MathNet.Numerics, but I keep getting the error "Matrix dimensions must agree 2x3".

/*****************************************************
    * MathNet.Numerics Multiple linear regression test *
    ****************************************************/

var ydata = new double[] { 15, 20, 25, 55, 95 };
var xdata = new[] { new[] { 1.0, 4.0 }, new[] { 2.0, 5.0 }, new[] { 3.0, 6.0 } };

var x = DenseMatrix.OfColumns(2, 3, xdata);
var y = new DenseVector(ydata);

var p = x.QR().Solve(y); // error: matrix dimensions must agree 2x3
var a = p[0];
var b = p[1];

MessageBox.Show(a.ToString(), "Test");
MessageBox.Show(b.ToString(), "Test");

Answer 1

The exception text is really bad in this case, we should fix this.

There are two concrete problems causing this to fail:

• The system with 3 unknowns but only 2 equations/samples (2x3 matrix) is under-defined; Applying a regression to such a problem does not actually make any sense as there is an infinite number of exact solutions.

• For these two samples/equations the right hand side y has actually 5 values (instead of 2, to match each sample)

Maybe it is easier to start with the new Fit class available in v3. For example, let's do an ordinary regression with the following samples:

a    b         y
1.0, 4.0  -->  15.0
2.0, 5.0  -->  20.0
3.0, 2.0  -->  10.0

With the problem to find u, v and w that minimize the error with these samples in

y ~= u + a*v + b*w

With the Fit class this would look like the following:

double[] p = Fit.MultiDim(
    new[] {new[] { 1.0, 4.0 }, new[] { 2.0, 5.0 }, new[] { 3.0, 2.0 }},
    new[] { 15.0, 20, 10 },
    intercept: true);
// p = [u,v,w] = [-1.25,1.25,3.75], hence y ~= -1.25 + 1.25*a + 3.75*b

With linear algebra and matrix decompositions the same could look like:

var A = DenseMatrix.OfArray(new[,] {
           { 1.0, 1.0, 4.0 },
           { 1.0, 2.0, 5.0 },
           { 1.0, 3.0, 2.0 }});
var b = new DenseVector(new[] { 15.0, 20, 10 });
var p2 = A.Solve(b);
// p = [-1.25,1.25,3.75]

Does that help?