Variance & Standard Deviation
Two classes both average 75 on one test, `[73, 74, 75, 76, 77]` and `[55, 65, 75, 85, 95]`. By that average they look identical, yet one class is ten times more spread out than the other.
- ▸Two classes with the same average read as identical, because the gaps from it always cancel to zero.
- ▸Squaring those gaps splits two equal-average classes a hundred to one, and one square root makes the number readable.
- ▸Subtract the center, divide by the spread, and no feature can bully a model just for being bigger.
Give two classes the same test. Class A scores [73, 74, 75, 76, 77]. Class B scores [55, 65, 75, 85, 95]. Both average to 75. Report only that average and the two classes are indistinguishable, yet one is packed tight around the middle and the other is flung wide. The average fixes where the data sits. It says nothing about how far the data strays.
The obvious repair is to measure how far. For each score, take its gap from the average, then average those gaps. Try it on Class A: the gaps are -2, -1, 0, +1, +2, and they sum to zero. Try Class B: the gaps are -20, -10, 0, +10, +20, and they also sum to zero. This is not a coincidence about these two datasets. It is the defining property of the average carried in from the previous topic: the average is the balance point, so the signed gaps around it always cancel, every time, for every dataset.
A signed gap from the average has a name, the deviation, and averaging the deviations is a dead end: the answer is always zero and tells you nothing. The fix is a single decision about what to do to each gap before you add it up, and that decision is the whole rest of this topic.