Maximum Likelihood Estimation
A coin came up heads `30` times in `100` tosses, but you never watched it get minted. Which bias best explains the flips you actually got, and why is the honest guess simply `30/100`?
- ▸The setting that best explains 30 heads in 100 flips is just
30/100, and this shows why. - ▸Multiply enough tiny probabilities and the product underflows to zero; the log rescues it without moving the answer.
- ▸Strip the prior out of Bayes and what's left, the best-fit knob, is exactly this method.
You are handed a record of 100 coin tosses: 30 came up heads, 70 tails. Somewhere there is a real coin with a real bias, the fraction of tosses it turns up heads, but nobody will tell you what it is. All you can do is guess a bias, ask how probable your exact record of 30 heads would be under that guess, and then try another guess.
Try a fair coin, a bias of 0.5. Thirty heads in a hundred is possible under a fair coin but on the low side, so the record scores a smallish probability. Now try a bias of 0.3. Suddenly 30 heads is right in this coin's wheelhouse, and the same record scores its highest probability of all. Push the guess down to 0.1 and 30 heads becomes a fluke again; the score collapses.
Sweep the guessed bias from 0 to 1 and the probability of your fixed record rises to a single peak, then falls away. That peak, the guess that makes the flips you actually got as probable as they can be, is your best estimate of the coin. And it lands exactly where you would have put it by hand: 30 heads over 100 tosses, a bias of 0.3. The whole payoff of the method is that it explains why the obvious count is the right one to trust.