Restrict, then renormalize
Before you know anything, the sample space is everything that could happen. Learning that B occurred does something specific and mechanical: every outcome outside B is now impossible, so you delete it. But the surviving outcomes no longer sum to 1, so you divide them all by how much survived.
- numerator
- The part of A that survived the restriction — outcomes where both A and B happen.
- denominator
- How much of the original space survived. This is the renormalization, and forgetting it is the most common error.
- P(B) > 0
- Required. Conditioning on an impossible event is undefined — there is nothing left to renormalize against.
A conditional is still a probability
Fix B and P(· | B) obeys every axiom: it's non-negative, P(Ω | B) = 1, and it adds over disjoint events. Everything you know about probability applies unchanged inside the restricted world.
The unconditional is conditional too
P(A) is really P(A | everything you already assume) — the sample space, the model, the population. There is no such thing as a probability without a reference class; some are just left implicit.
Conditioning can go either way
Learning B can raise P(A), lower it, or leave it alone. All three are ordinary. The third case has a name — independence — and gets section 04.
Which side of the bar?
Most errors in applied probability are translation errors — turning an English sentence into the wrong conditional. The fix is a habit: identify the reference class first, because that's what goes to the right of the bar.
| English | Reference class | Notation |
|---|---|---|
| "Of the patients who tested positive, how many are ill?" | Positive testers | P(ill | positive) |
| "Of the ill patients, how many test positive?" | Ill patients | P(positive | ill) |
| "Among smokers, the rate of X is…" | Smokers | P(X | smoker) |
| "Given that it rained, …" | Rainy days | P(· | rain) |
| "If the coin is fair, the chance of …" | Fair-coin worlds | P(· | fair) |
| "The false positive rate" | Healthy people | P(positive | healthy) |
| "The chance this positive is a false alarm" | Positive testers | P(healthy | positive) |
| "X% of accidents involve Y" | Accidents | P(Y | accident) |
| "Y raises accident risk by X%" | People with Y | P(accident | Y) |
The same equation, rearranged
Multiply the definition through by the denominator and you get the multiplication rule — a way to build joint probabilities out of conditionals. This rearrangement is quietly one of the most useful facts in machine learning.
Multiplication rule
Two decompositions of the same joint probability. Setting them equal to each other and dividing gives Bayes' theorem in one line — see section 08.
Chain rule
Apply the multiplication rule repeatedly and any joint distribution over any number of variables factorizes into a product of conditionals — each one conditioned on everything before it. The factorization is exact, always, with no assumptions.
# the chain rule, made literal P("the", "cat", "sat") = P("the") × P("cat" | "the") × P("sat" | "the", "cat") # an n-gram model truncates the history — an approximation P("sat" | "the", "cat") ≈ P("sat" | "cat") # a transformer keeps all of it, up to the context window — no truncation
When knowing changes nothing
Independence is the special case where conditioning is a no-op. Learning B leaves your belief about A exactly where it was.
Three equivalent statements
Any one implies the other two. The third is usually taken as the formal definition because it's symmetric and stays valid when a probability is zero.
In the hero square, independence is exactly when the two horizontal dividers line up — the split of A is the same whichever column you're in. Try the "make independent" preset and watch them level.
Pairwise ≠ mutual
Three events can be independent in every pair and still not jointly independent. The classic case: two fair coin flips plus the event "the flips matched." Any two of those three are independent; all three together are not, because any two determine the third completely.
Independence is an assumption, not an observation
It is almost never verified in practice — it's assumed because it makes the maths tractable, and the assumption is usually somewhat false. Knowing which independence assumptions your model makes is knowing where it will fail.
Independent once you know something else
The more useful and more subtle relative. A and B may be strongly dependent overall, yet become independent once you fix a third variable C.
The common-cause case
Ice cream sales and drowning deaths are strongly correlated. Condition on the temperature and the correlation vanishes — each was driven by the season, not by the other. C explains the dependence, and once you hold it fixed, nothing is left.
| Structure | Shape | Unconditionally | Conditioning on C |
|---|---|---|---|
| Chain | A → C → B | Dependent | Blocks the path — becomes independent |
| Fork · common cause | A ← C → B | Dependent | Blocks the path — becomes independent |
| Collider · common effect | A → C ← B | Independent | Opens the path — becomes dependent |
These three rules are the whole of d-separation, which is how graphical models read independence relations straight off a diagram. "Control for everything you can" is bad advice precisely because of row three.
P(A | B) is not P(B | A)
The most consequential fact in this document. These two numbers can differ enormously, and confusing them has sent people to prison and sent patients into unnecessary treatment. The unit square makes it obvious why: one is a fraction of a column, the other a fraction of a band.
A test for a rare condition
Building the whole from the cases
If you can carve the world into cases that don't overlap and cover everything, then the overall probability of anything is the weighted average of its probability within each case.
The two rules that make trees work: multiply along a path because each step is a conditional given the ones before it, and add across paths because the leaves are mutually exclusive. Every leaf is a joint probability, and they sum to 1.
In the hero square, this is just the observation that the total A area is the left A rectangle plus the right A rectangle — each one a conditional height times a column width.
The inversion, derived in two lines
Bayes' theorem gets its own entry in the checklist, but it isn't a new idea — it's the definition of conditional probability written twice and rearranged. Worth seeing that it costs nothing to derive.
That's it. The theorem is a bookkeeping identity. What makes it philosophically interesting is the interpretation laid on top: P(A) as a prior belief, P(B | A) as how well the hypothesis predicts the evidence, and P(A | B) as the updated belief.
Where intuition reliably fails
Monty Hall
Three doors, one prize. You pick one. The host — who knows where the prize is and will always open a losing door — opens one of the others. Switching wins two times in three.
The resolution is that the host's action is constrained, so it carries information. Your original door was right 1/3 of the time and that never changes; the remaining 2/3 gets concentrated onto the single unopened door. Change the rule so the host opens a door at random and might reveal the prize, and switching becomes worthless. The conditioning depends on the protocol, not just the outcome.
Two children
"A family has two children. At least one is a boy. What's the chance both are?" The textbook answer is 1/3 — of the four equally likely orderings, three contain a boy, and one of those is two boys.
But it depends entirely on how you learned it. If you met one child at random and he was a boy, the answer is 1/2. Same words, different sampling process, different conditioning set. This problem is famous less as a puzzle than as a demonstration that you cannot condition on a statement — only on an event, and the event includes how the information reached you.
Simpson's paradox
A treatment can have a higher success rate in every subgroup and a lower success rate overall. Not a trick — a genuine arithmetic possibility, arising when group sizes are unequal and the grouping variable also influences the outcome.
The famous real case is the 1973 Berkeley graduate admissions data, where the university-wide rate favoured men while most individual departments favoured women — because applicants sorted unevenly across departments with very different competitiveness. The paradox is that neither number is wrong. They answer different questions, and deciding which to report is a causal judgement, not a statistical one.
Where it actually shows up
| Where | The conditional | What it means |
|---|---|---|
| Classification | P(y | x) | Literally the model's output. A classifier is a conditional distribution over labels given features |
| Discriminative models | P(y | x) | Logistic regression, most neural nets — model the conditional directly |
| Generative models | P(x | y) P(y) | Naive Bayes, GDA — model how each class generates data, then invert with Bayes |
| Naive Bayes | ∏ P(xⱼ | y) | The conditional independence assumption, stated as a product |
| Language models | P(wₜ | w₁…wₜ₋₁) | The chain rule factorization of a sentence's joint probability |
| Cross-entropy loss | −log P(y | x) | The training objective is the negative log of a conditional probability |
| Graphical models | Conditional independence | The graph structure is a set of conditional independence claims |
| Reinforcement learning | P(s' | s, a) | Transition dynamics — the next state given the current state and action |
| VAEs | q(z | x), p(x | z) | Encoder and decoder are both conditional distributions |
| Diffusion models | p(xₜ₋₁ | xₜ) | Each denoising step is a conditional |
| Causal inference | P(Y | X) vs P(Y | do(X)) | Observing versus intervening — different quantities, and only the second answers "what if we acted" |
| Fairness criteria | P(ŷ | y, group) | Most fairness definitions are statements about conditionals holding across groups |
Drills that build the instinct
| # | Do this | You'll know it when |
|---|---|---|
| 1 | For every conditional you meet this week, say out loud "out of all the ___, what fraction are ___" | You catch a reversed conditional in someone else's writing |
| 2 | Work the medical test problem with pen and paper at three prevalences before touching the slider | You can predict roughly where PPV lands without computing it |
| 3 | Build a 2×2 contingency table from counts and compute all four conditionals plus both marginals | You stop needing the formula and just read the table |
| 4 | Simulate Monty Hall 10,000 times in ten lines of code | The 2/3 stops feeling like a trick |
| 5 | Construct a dataset that exhibits Simpson's paradox from scratch | You understand it as arithmetic rather than as a curiosity |
| 6 | Simulate collider bias — two independent variables, condition on their sum being large | You've seen conditioning manufacture a correlation from nothing |
| 7 | Write the chain rule factorization of a five-word sentence by hand | The link between conditioning and language modeling is concrete |
| 8 | Take one published statistic and identify its implicit reference class | You notice how often the reference class is unstated or wrong |
The one-paragraph summary
Conditional probability is the operation of restricting the sample space to the outcomes consistent with what you've learned, then renormalizing so the survivors sum to one — which is exactly what P(A | B) = P(A ∩ B) / P(B) says. Rearranged, it gives the multiplication rule and hence the chain rule, which factorizes any joint distribution into a product of conditionals and is the reason next-token prediction constitutes language modeling. Independence is the case where conditioning changes nothing, and it is emphatically not the same as mutual exclusivity. Conditional independence — independence once a third variable is fixed — is the more useful notion, underpinning naive Bayes and graphical models, and it comes with the warning that conditioning on a common effect creates dependence rather than removing it. The single most costly error is confusing P(A | B) with P(B | A), which base rates can separate by orders of magnitude; recasting the problem as counts out of a concrete population is the most reliable defence. And every dataset is already conditioned on how it was collected, so the quantity you compute is never quite the quantity you wanted.