HackerWhere does learning get a signal without an answer key?
Start with an unlabeled patch of an image, then switch to a game that requires a sequence of actions. Neither has an answer key, but finding structure and choosing a move need different signals.
Questions in this reading · 7
- What can images teach without labels?
- Where does the learning signal come from?
- Why can’t a policy classify each move in isolation?
- Where does AlphaGo get experience?
- Where do strategies absent from demonstrations come from?
- How do game agents improve beyond their starting point?
- What must be recovered behind the mask?
What can images teach without labels?
Unsupervised learning looks for structure in observations. Learning features from image patches shows what that structure can look like before labels enter the task.
Compare original passages and timestamps · 2
I'm not going to tell you what the right answer isfind a better way to represent images than the raw input pixelsWhere does the learning signal come from?
Finding patterns in unlabeled data and choosing actions from rewards both dispense with a teacher giving the right answer. Only the latter must account for the consequences of a decision.
Compare original passages and timestamps · 2
I'm not going to tell you what the right answer isthere's no supervisor there's just this reward signalWhy can’t a policy classify each move in isolation?
An action changes the next situation and thus the choices and rewards still available. The dependence is temporal even when the delay is short.
Compare original passages and timestamps · 2
the consequence of trashing it may not happen until much laterthe agent is actually influencing the data that it seesWhere does AlphaGo get experience?
Describe the task, experience and performance measure before examining the algorithm. AlphaGo provides distinct examples of experience from human games and self-play.
Compare original passages and timestamps · 2
a computer program is set to learn from an experiencethrough self-playing, reinforcement learning, it plays against different versions of itselfWhere do strategies absent from demonstrations come from?
Compare self-play in Go with exploration in language-model problem solving. Trial and error can find new strategies when feedback distinguishes useful attempts.
Compare original passages and timestamps · 2
through self-playing,
reinforcement learning, it plays against different versions
of itself many millions of times,we're going to check them and we're going to do more of what workedHow do game agents improve beyond their starting point?
AlphaGo imitates games and then plays against itself. MarI/O selects and reproduces networks according to game progress.
Compare original passages and timestamps · 2
through self-playing, reinforcement learning, it plays against different versions of itselfOnly the neural networks that produce the highest fitness are selectedWhat must be recovered behind the mask?
Mask part of a tree. Recovering every pixel and recovering an encoding define different training tasks. LeCun uses this distinction to explain JEPA: the full observation is encoded too, and its representation supplies the prediction target. A predictor need not chase every leaf’s incidental motion, but accuracy cannot restore distinctions discarded by the representation itself. Identify the space in which the error is computed.
Passages and timestamps · 2
you’re only trying to predict an abstract representation of the inputsyou’re not going to have to predict every single pixel of every leafTry it on your own example
If you change one action but leave the current picture unchanged, which tasks can still have a different outcome?