Backpropagation and batch training
-
22-09-2019 - |
문제
Backpropagation calculates dW
(weight delta) per weight per pattern, so it's straightforward how to modify weights when doing stochastic training. How do I use it for batch training, though? Simply accumluate dW
over the entire training set and then apply the modfication, or is there more to it?
해결책
Yes, just accumluate dW over the entire training set. At least that is how I coded it back in grad school...
다른 팁
You can do a lot with the different gradients from the different samples. That includes higher order information (approximate 2nd derivative) or conjugate gradient or natural gradient or ... :)
제휴하지 않습니다 StackOverflow