skipnothing/Math for Machine Learning

Bayes' Theorem

A test catches 80% of a disease, you test positive, and it seems to mean an 80% chance of being sick. The real number, `P(disease | positive)`, can sit near 7.5%, and one line of algebra says why.

  • The famous formula is only the product rule rearranged: no new math, just one division.
  • An 80% hit-rate test can still leave you 7.5% likely to be sick, because the disease's rarity sets the floor.
  • A second positive result runs on the belief the first one already left, not the original odds.
BUILDS ON
01

Divide the Product Rule

A screening test is easy to describe in one direction. Of people who have a disease, 80% test positive, that is the test's hit rate, printed on the box. But a patient holding a positive result wants the other direction: given this positive, how likely is the disease? Those are the two conditionals from the last topic, sitting on one cell, and they are not the same number.

The bridge between them is already in your hands. The product rule said a joint probability factors two ways, and both land on the same cell:

P(positive | disease) · P(disease) = P(disease | positive) · P(positive)

Both sides equal the chance of being sick and testing positive. To get the direction you want alone, divide both sides by the chance of a positive result:

P(disease | positive) = P(positive | disease) · P(disease) / P(positive)

That is the whole formula. There is no new mathematics in it, only the product rule with one division applied. What it buys is the flip: it takes the direction the test hands you and turns it into the direction you actually care about.

REARRANGE THE IDENTITY INTO THE FORMULA

Keep going, sign up to unlock the rest

4 more parts in this topic, plus 19+ more topics in Math for Machine Learning.

Sign up, it's freeSee the full Math for Machine Learning
Probability0/4#17 Joint & Conditional Probability
#19 Maximum Likelihood Estimation