HackerGradient descent, how neural networks learn | Deep Learning Chapter 2
3Blue1BrownYouTube ↗
Further reading 1
Caption notes
Averaging costs over training images defines the landscape to optimize. The negative gradient points downhill locally; one downhill step does not validate every derivative.
Creator’s caption track ↗Original passages · 2
So then what you do is consider the average cost over all of the tens of thousands of training examples at your disposal.This process of repeatedly nudging an input of a function by some multiple of the negative gradient is called gradient descent.