Vectorization of a gradient descent code

Question

I am implementing a batch gradient descent on Matlab. I have a problem with the update step of theta. theta is a vector of two components (two rows). X is a matrix containing m rows (number of training samples) and n=2 columns (number of features). Y is an m rows vector.

During the update step, I need to set each theta(i) to

theta(i) = theta(i) - (alpha/m)*sum((X*theta-y).*X(:,i))

This can be done with a for loop, but I can't figure out how to vectorize it (because of the X(:,i) term).

Any suggestion?

Answer 1

Looks like you are trying to do a simple matrix multiplication, the thing MATLAB is supposedly best at.

theta = theta - (alpha/m) * (X' * (X*theta-y));

Answer 2

In addition to the answer given by Mad Physicist, the following can also be applied.

theta = theta - (alpha/m) * sum( (X * theta - y).* X )';