What Is Probability?

What Does Probability Mean?

When we say “there is a 60% chance of rain tomorrow,” what do we really mean? It depends on who you ask. There are two main ways to interpret probability, and both are useful in different contexts.

The frequency interpretation says that probability is the limiting proportion of times an event would occur if you repeated an experiment infinitely many times. Flip a fair coin 10,000 times, you’ll see roughly 5,000 heads, or a frequency of about 0.5. The more times you repeat the experiment, the closer your observed frequency tends toward the true probability. This interpretation works beautifully for games of chance and for modeling repeatable financial events like daily stock returns.

The subjective interpretation treats probability as a degree of belief. A quant might say “I assign a 30% probability to this trade being profitable”, not because they ran an experiment, but because they’ve formed a belief based on analysis, experience, and available information. Subjective probability is the foundation of Bayesian statistics, where your beliefs are updated as new evidence arrives.

Both views are valid, and you will encounter both in quantitative finance. The mathematics of probability, the rules, formulas, and theorems, works the same way regardless of which interpretation you prefer.

Sample Spaces and Events

Before we can talk about probability, we need to define what we are measuring.

The sample space $\Omega$ is the set of all possible outcomes of a random experiment. If you roll a six-sided die, the sample space is $\Omega = \{1, 2, 3, 4, 5, 6\}$. If you observe the daily return of a stock, the sample space is all real numbers (though in practice, you might restrict to a range like $[-0.5, 0.5]$ for a single day).

An event is a subset of the sample space. “Rolling an even number” is the event $A = \{2, 4, 6\}$. An event can contain one outcome, many outcomes, or no outcomes at all (the empty set $\emptyset$).

The Probability Axioms

Kolmogorov’s axioms, laid out by Russian mathematician Andrey Kolmogorov in 1933, form the foundation of modern probability theory. They are surprisingly simple.

First, probability is always between 0 and 1:

Second, the probability that something happens at all, that you land in the sample space, is 1:

Third, for any two events that cannot happen simultaneously (called disjoint or mutually exclusive events), the probability that either one happens is the sum of their individual probabilities:

From these three axioms, almost everything else in probability follows. For instance, if $A$ and $A^c$ (the complement of $A$, meaning “not $A$”) are complementary events, then $P(A) + P(A^c) = 1$. This makes intuitive sense: either $A$ happens, or it doesn’t, those are the only possibilities, and their probabilities must sum to 1.

Complementary Events

The complement rule is one of the most practical shortcuts in probability. If you know the probability of a market crash in a given year is 5%, then the probability of no crash is 95%. Rather than trying to enumerate all the ways a crash might not happen, you can often find the probability of the complement and subtract from 1.

Addition and Multiplication Rules

For non-disjoint events, events that can happen together, the addition rule needs a correction:

The term $P(A \cap B)$ subtracts the overlap that was counted twice. Imagine counting students who take math or economics: if you add the math students to the economics students, students taking both get counted twice, so you subtract them once.

The multiplication rule gives the probability that two events both happen:

Here $P(B|A)$ is the conditional probability of $B$ given that $A$ has occurred, we will look at this next.

If $A$ and $B$ are independent (meaning knowing whether $A$ happened tells you nothing about whether $B$ happened), then $P(B|A) = P(B)$ and the multiplication rule simplifies to:

Conditional Probability

Often we want to know the probability of an event given that we already know something else has happened. This is conditional probability:

Read this as “the probability of $A$ given $B$.” For example, if you draw a card from a standard deck and want to know the probability it is a heart given that you see a red card, you are looking for $P(\text{heart} | \text{red})$.

Venn diagrams are a helpful visual tool here. Imagine a rectangle representing the sample space, with two overlapping circles representing events $A$ and $B$. The overlap, the intersection, is $A \cap B$. Conditional probability $P(A|B)$ asks: given that we are inside circle $B$, what fraction of $B$ is also inside $A$? That fraction is exactly $\frac{P(A \cap B)}{P(B)}$.

Independence

Two events are independent if knowing one does not change the probability of the other:

From this, the multiplication rule for independent events follows directly:

A classic example: if you flip a fair coin twice, the result of the first flip does not affect the second. $P(\text{heads on flip 2} | \text{heads on flip 1}) = 0.5 = P(\text{heads on flip 2})$.

Simulating Probability in Python

Python is an excellent tool for building intuition about probability through simulation. The law of large numbers, which we will study later, says that the more times you repeat an experiment, the closer your observed frequencies get to the true probabilities. Let’s demonstrate this.

import numpy as np
import matplotlib.pyplot as plt

np.random.seed(42)


n_flips = 10_000
flips = np.random.choice([0, 1], size=n_flips)  # 0 = tails, 1 = heads

# Calculate running proportion of heads
running_prop = np.cumsum(flips) / np.arange(1, n_flips + 1)

plt.figure(figsize=(10, 5))
plt.plot(np.arange(1, n_flips + 1), running_prop, alpha=0.7)
plt.axhline(y=0.5, color='red', linestyle='--', label='True probability = 0.5')
plt.xlabel('Number of flips')
plt.ylabel('Proportion of heads')
plt.title('Law of Large Numbers: Coin Flip Simulation')
plt.legend()
plt.grid(alpha=0.3)
plt.show()

# Simulate rolling a fair six-sided die 10,000 times
n_rolls = 10_000
rolls = np.random.choice([1, 2, 3, 4, 5, 6], size=n_rolls)

print(f"Observed frequencies:")
for face in range(1, 7):
    freq = np.mean(rolls == face)
    print(f"  Face {face}: {freq:.4f} (true probability: {1/6:.4f})")

PROMPT

Use numpy and matplotlib to simulate coin flips and show the running proportion of heads converging to 0.5.

This code demonstrates that as the number of trials grows, the observed proportion converges toward the true probability. Try varying n_flips to see how convergence behaves with different sample sizes.

References

Ross, S. M. A First Course in Probability, Chapter 1. Pearson. Grimmett, G. R. and Welsh, D. Probability, Chapter 1. Oxford University Press.