skipnothing/Math for Machine Learning
UNIT 04

Probability

Bayes, conditional probability, and maximum likelihood, reasoning under uncertainty

ML is fundamentally about making decisions under uncertainty. Probability gives you the language to reason about it precisely.

Loading…
BUILDS ON
TOPICS
#17

Joint & Conditional Probability

Shrink a two-way table of emails to one row or column and read joint, marginal, and conditional probability off the same cells by swapping the denominator, factor any joint two ways with the product rule, test independence by checking whether the conditional equals the marginal, and see why `P(spam | free money)` and `P(free money | spam)` are two different numbers.

Your spam filter flags an email holding `free money` and calls it almost certainly spam. Ask the mirror question, how often spam even contains `free money`, and that number comes back small: the same evidence reads two different ways.

14 min
#18

Bayes' Theorem

Rearrange the product rule into Bayes' theorem, `P(A|B) = P(B|A)·P(A)/P(B)`, read its four pieces (prior, likelihood, posterior, evidence) off a medical-test grid, build the evidence from the law of total probability, watch a rare base rate drag an 80%-accurate positive down to a 7.5% chance of disease, and feed each posterior back as the next prior.

A test catches 80% of a disease, you test positive, and it seems to mean an 80% chance of being sick. The real number, `P(disease | positive)`, can sit near 7.5%, and one line of algebra says why.

16 min
#19

Maximum Likelihood Estimation

Slide a single knob until the data you already collected becomes as probable as it can be, and read that peak as the estimate: a coin's bias is heads over flips, a bell's centre is the sample average. See why likelihood fixes the data and varies the parameter, why the log-likelihood shares the same peak while surviving huge datasets, and why the whole method is Bayes' theorem with the prior deleted.

A coin came up heads `30` times in `100` tosses, but you never watched it get minted. Which bias best explains the flips you actually got, and why is the honest guess simply `30/100`?

18 min
#20

Central Limit Theorem

Average random samples from any lopsided population and pile the averages into a bell centred on the true mean, set its width with `σ/√n` so four times the data only halves it, and separate the three distributions in play so only the sample averages turn normal, never the raw data.

Take a lopsided pile of household incomes, grab `5` at random and average them, then repeat a thousand times. Those averages stack into a clean bell, and each extra draw buys less width than the one before.

12 min
All units