Learn Bayes' Theorem

Update beliefs based on evidence

Calculator Dice Coins Marbles Cards Learn

The Formula

The Components

P(A|B)

Posterior

Probability of A given B

P(A)

Prior

Initial belief before evidence

P(B|A)

Likelihood

Probability of evidence if A is true

P(B)

Marginal Likelihood

Total probability of seeing B

In Plain English

Bayes' theorem answers: "Given that I observed evidence B, what is the probability that hypothesis A is true?"

It combines:

- Your prior belief about A
- How likely B would be if A were true
- How likely B is overall

Calculating P(B) - The Denominator

Often we need to calculate P(B) using the Law of Total Probability:

This gives us the full Bayes formula:

Note: is the probability that A is NOT true.

Classic Example: Medical Testing

The Problem

A disease affects 1% of the population. A test has:

- 95% sensitivity (true positive rate)
- 95% specificity (true negative rate)

If you test positive, what is the probability you have the disease?

Identify the probabilities

(1% prevalence)
(sensitivity)
(false positive rate)

Calculate P(positive)

Apply Bayes' Theorem

Surprising Result!

Even with a 95% accurate test, a positive result only means a 16% chance of having the disease! This is because the disease is rare (1%), so most positive tests are false positives.

The Base Rate Fallacy

The base rate fallacy occurs when people ignore the prior probability (base rate) and focus only on the test accuracy.

Intuitive Explanation

Imagine testing 1,000 people:

People with disease:1,000 × 1% = 10

People without disease:1,000 × 99% = 990

True positives (sick, test +):10 × 95% = 9.5

False positives (healthy, test +):990 × 5% = 49.5

P(disease | positive) =9.5 / 59 = 16.1%

Key Lesson

When the base rate is low, even accurate tests produce many false positives. Always consider how common the condition is before interpreting test results!

When to Use Bayes' Theorem

Use Bayes When...

- You have prior beliefs about hypotheses
- You observe new evidence
- You want to update beliefs based on evidence
- You know P(B|A) but need P(A|B)
- Medical/diagnostic test interpretation
- Spam filtering, fraud detection

Real-World Applications

Medicine: Interpreting test results
Email: Spam filtering
Finance: Fraud detection
Law: DNA evidence evaluation
AI: Machine learning classifiers
Science: Hypothesis testing

Terminology Quick Reference

Term	Symbol	Meaning
Prior		Initial belief before seeing evidence
Posterior		Updated belief after seeing evidence
Likelihood		Probability of evidence if hypothesis true
Marginal		Total probability of observing evidence
Sensitivity		True positive rate (medical tests)
Specificity		True negative rate (medical tests)
PPV		Positive predictive value (what Bayes calculates)

Common Mistakes to Avoid

Confusing P(A|B) with P(B|A)

P(positive | disease) is NOT the same as P(disease | positive). A test being 95% accurate does not mean a positive result = 95% chance of disease.

Ignoring the Prior (Base Rate)

Even a very accurate test has limited value if the condition is very rare. Always consider how common the hypothesis is before testing.

Forgetting to Update

With multiple pieces of evidence, use the posterior from one update as the prior for the next. This is called sequential Bayesian updating.

Wrong P(B|A')

The false positive rate P(B|A') is NOT equal to 1 - P(B|A). Sensitivity and false positive rate are independent parameters.

Try These Scenarios

Loaded Die Detection

Is the die loaded after rolling three 6s?

Biased Coin Test

Is the coin biased after 8 heads in 10 flips?

Classic Two-Bag Problem

Which bag did I pick based on the marble drawn?

Rigged Deck Detection

Is the deck rigged after drawing 3 aces?

Frequently Asked Questions

Quick Reference

Posterior = Updated Belief

Prior = Initial Belief

Likelihood = P(evidence | hypothesis)