Introduction to Machine Learning
How Machines Learn?
Video 1 of 2
Before looking at any particular algorithm, it helps to ask a simpler question: where does the learning signal come from? Almost every machine learning system in existence answers that question in one of three ways—and most real systems mix them.
A machine can learn from observation, by measuring how variables in a dataset move together. It can learn from consequence, by acting, being rewarded or punished, and adjusting. Or it can learn from knowledge we hand it directly, as rules and logical relationships it applies and combines. These three sources—data, experience, and rules—are the subject of the sections below.
Keep these three in mind as you read the rest of this article, because everything that follows is an implementation of one of them rather than a new kind of learning:
- Learning Paradigms (section 2) is the taxonomy you will meet everywhere else—supervised, unsupervised, reinforcement. It cuts the same territory a second way: by what the training data looks like, rather than by where the signal comes from.
- Deterministic Models and Probabilistic Models (sections 3 and 4) are the classical machinery of learning from data. They differ in whether they answer with a single value or with a distribution, not in where their knowledge comes from.
- Optimization Algorithms (section 5) are how a model actually gets fitted once you have chosen one—including the derivative-free methods that make learning from experience tractable.
- Symbolic AI (section 6) is learning and reasoning with rules, made concrete.
- Deep Learning (section 7) is often presented as a fourth paradigm. It isn't. A neural network is a very flexible function trained by the same three signals—mostly statistical, sometimes rewarded, occasionally fused with symbolic reasoning. What changes is not the source of the signal but how much of the work the machine does for itself.