Bayes' Theorem and Conditional Probability

Recap: Conditional Probability

Before diving into Bayes’ theorem, let us revisit the conditional probability formula from the first article:

This reads as “the probability of $A$ given $B$ equals the joint probability of $A$ and $B$, divided by the probability of $B$.” The intuition: restricting our attention to the world where $B$ has happened, what fraction of those outcomes also have $A$?

By rearranging this definition, we get the multiplication rule:

Both versions are equivalent. Which one you start from depends on what you know and what you are trying to find.

Bayes’ Theorem

Bayes’ theorem is simply a rearrangement of the multiplication rule, but its implications are enormous:

In words:

• Prior $P(A)$: Your initial belief about $A$ before seeing any evidence.
• Likelihood $P(B|A)$: How likely would this evidence $B$ be, if $A$ were true?
• Posterior $P(A|B)$: Your updated belief about $A$ after seeing evidence $B$.
• Normalizing constant $P(B)$: Ensures the posterior is a valid probability (sums to 1 over all possible hypotheses).

The normalizing constant is calculated as:

This is just the law of total probability: the total probability of observing evidence $B$ is the weighted average of the likelihood under each possible hypothesis.

A Concrete Example: Earnings Surprise

Suppose you are analyzing a stock. Let $U$ be the event “the company reports positive earnings surprise” and let $R$ be the event “the stock goes up the next day.”

You have historical data telling you that:

• $P(U) = 0.4$ (40% of quarters have a positive surprise)
• $P(R) = 0.5$ (the stock goes up 50% of trading days)
• $P(R|U) = 0.75$ (given a positive surprise, the stock goes up 75% of the time)

What is $P(U|R)$ the probability of a positive earnings surprise given that the stock went up?

So given that the stock went up, there is a 60% chance there was a positive earnings surprise, higher than the base rate of 40%. The evidence has updated our belief upward.

Using a 2×2 Table for Intuition

The table shows what happens if we observe 100 days. From the marginals, 40 days have a surprise and 50 days have a stock up day. The joint count of “surprise AND stock up” is $0.75 \times 40 = 30$. The conditional $P(U|R) = 30/50 = 0.60$, exactly as Bayes’ theorem computed.

Conjugate Priors

A powerful practical simplification arises with conjugate priors. If your prior and posterior are from the same family of distributions, then updating is computationally trivial.

The Beta-Binomial conjugate is a classic example. Suppose you observe $k$ successes in $n$ Bernoulli trials (like $k$ up-days in $n$ trading days). If your prior on the success probability $\theta$ is Beta($\alpha, \beta$), then the posterior is also Beta:

You simply add the observed counts to the prior hyperparameters. No complicated integration required.

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats


alpha_prior, beta_prior = 2, 2

# Observed data: 6 up days out of 10 trading days
k = 6
n = 10

# Posterior: Beta(alpha + k, beta + n - k)
alpha_post = alpha_prior + k
beta_post = beta_prior + n - k

# Plot prior and posterior
theta = np.linspace(0, 1, 1000)
prior_pdf = stats.beta.pdf(theta, alpha_prior, beta_prior)
post_pdf = stats.beta.pdf(theta, alpha_post, beta_post)

plt.figure(figsize=(10, 5))
plt.plot(theta, prior_pdf, 'b--', lw=2, label=f'Prior Beta({alpha_prior},{beta_prior})')
plt.plot(theta, post_pdf, 'r-', lw=2, label=f'Posterior Beta({alpha_post},{beta_post})')
plt.axvline(x=0.6, color='gray', linestyle=':', label='MPE = 0.6')
plt.xlabel('θ (probability of up day)')
plt.ylabel('Density')
plt.title('Bayesian Update: Beta-Binomial Conjugate')
plt.legend()
plt.grid(alpha=0.3)
plt.show()

# Posterior mean (Bayesian estimate of theta)
posterior_mean = alpha_post / (alpha_post + beta_post)
print(f"Prior mean:   {alpha_prior / (alpha_prior + beta_prior):.4f}")
print(f"Posterior mean: {posterior_mean:.4f}")
print(f"MLE (k/n):    {k / n:.4f}")

PROMPT

Use numpy, matplotlib, and scipy.stats to do a Bayesian update of a Beta prior with binomial data, plot the prior and posterior, and print the means.

The posterior mean (0.57) is a compromise between the prior mean (0.5) and the maximum likelihood estimate (0.6). The more data you observe, the more the posterior shifts toward the data.

Applications in Quantitative Finance

Bayesian methods appear throughout quant finance. In signal processing, you might use a Kalman filter to update your estimate of an asset’s true price as new trades occur, each new price observation is evidence that updates your belief via Bayes’ theorem. In machine learning, naive Bayes classifiers use Bayes’ theorem to classify documents or market regimes based on observed features. In risk management, Bayesian updating allows you to incorporate new information (a credit downgrade, a macro shock) into your prior estimates of default probability.

The Base Rate Fallacy

Bayes’ theorem also explains a famous cognitive error: the base-rate fallacy. People tend to anchor on the accuracy of new evidence (a test, a signal, a diagnosis) while ignoring how rare the underlying event actually is. The result is a posterior that is wildly different from what intuition expects.

Consider a screening test for a rare disease affecting 1 in 1,000 people in the general population. The test is 99% accurate in both directions: it correctly identifies 99% of people who have the disease (sensitivity) and correctly clears 99% of healthy people (specificity). Let $D$ denote “has the disease” and $+$ denote “tests positive.” Translating the givens:

• $P(D) = 0.001$ base rate (prior)
• $P(+|D) = 0.99$ sensitivity (true positive rate)
• $P(+|\neg D) = 0.01$ false positive rate

We want $P(D|+)$ the probability a person actually has the disease given a positive test. First, by the law of total probability:

Then by Bayes’ theorem:

So even with a 99%-accurate test, only about 9% of people who test positive actually have the disease. The naive intuition “99% accurate test → 99% chance of having the disease” ignores that the disease is so rare that false positives vastly outnumber true positives. This is the base-rate fallacy in action, and it shows up everywhere, medical testing, fraud detection, hiring screens, and trading signals, wherever the prior probability is small.

References

Gelman, A. et al. Bayesian Data Analysis, Chapter 1. CRC Press. Ross, S. M. A First Course in Probability, Chapter 3. Pearson.