What exactly does delta mean in the gradient descent algorithm?
-
15-02-2021 - |
Question
As on the picture:
Could someone help me understand what exactly what delta means in the gradient descent algorithm?
Solution
The term is a derivative with respect to the theta 0
.
- Mark
theta
as coordinate on X-axis (let it be A) - Find corresponding coordinate on Y-axis (let it be B) so the point belongs to the function J
- Draw tangent line to that function at the point (A, B)
- The derivative is the slope of this tangent line.
The derivative is used to control two aspects of the cost function (J function) minimization:
- direction - sign of the slope tells you in which direction you should move along the X-axis in order to converge J
- rate - magnitude of the slope tells you how fast you should move
OTHER TIPS
This is a partial derivative with respect to theta_0.
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow