Overshooting
lastmod 2019-05-09created 2019-05-095 edits
TODO: Is it different from "Gradient Exploding"?
About
Pythonkim.tistory.com_-_Overshooting.png
This figure shows the case where the Learning rate is too large, illustrating the overshooting phenomenon. If the step size is too large, it can pass the lowest point and move to the opposite slope, and in severe cases it can even climb up the slope instead of descending it. Overshooting refers to the phenomenon of climbing up the slope and eventually escaping the bowl.