You should provide more details about yout problem - what exactly are you asking about - are we talking about linear regression in one or many dimensions? Simple or generalized ones?
In general, why do people use the GD?
- it is easy to implement
- it is very generic optimization technique - even if you change your model to the more general one, you can stil use it
So what about analytical solutions? Well, we do use them, your claim is simply false here (if we are talking in general), for example the OLS method is a closed form, analytical solution, which is widely used. If you can use the analytical solution, it is affordable computationaly (as sometimes GD is simply cheapier or faster) then you can, and even should - use it.
Neverlethles this is always a matter of some pros and cons - analytical solutions are strongly connected to the model, so implementing them can be inefficient if you plan to generalize/change your models in the future. They are sometimes less efficient then their numerical approximations, and sometimes there are simply harder to implement. If none of above is true - you should use the analytical solution, and people do it, really.
To sum up, you rather use GD over analytical solution if:
- you are considering changes in the model, generalizations, adding some more complex terms/regularization/modifications
- you need a generic method because you do not know much about the future of the code and the model (you are only one of the developers)
- analytical solution is more expensive computationaly, and you need efficiency
- analytical solution requires more memory, which you do not have
- analytical solution is hard to implement and you need easy, simple code