Gradient descent, how neural networks learn | Deep Learning Chapter 2

So then what you do is consider the average cost over all of the tens of thousands of training examples at your disposal.

This process of repeatedly nudging an input of a function by some multiple of the negative gradient is called gradient descent.