Random Variables and Probability Distributions

From Outcomes to Numbers

When you flip a coin, the outcome is “heads” or “tails”, words, not numbers. A random variable transforms outcomes like these into numerical values so we can do mathematics with them.

A random variable $X$ is a function that assigns a real number to each outcome in a sample space. You can think of it as a rule for measuring something random. The notation $X = x$ means “the random variable $X$ takes the value $x$.”

There are two fundamentally different types.

Discrete random variables take values in a countable set, typically whole numbers. The number of heads in 10 coin flips is discrete. The number of trades executed in a day is discrete. These variables have a probability mass function (PMF) that tells us $P(X = x)$ for each possible value.

Continuous random variables can take any value within a range. The daily return of a stock, the temperature tomorrow, the price of an option, these are all continuous. For continuous variables, the probability of landing on any exact value is technically zero; instead, we work with intervals and use a probability density function (PDF).

Probability Mass Functions

For a discrete random variable, the probability mass function (PMF) gives the probability of each outcome:

The function must satisfy two properties: $p(x) \geq 0$ for all $x$, and the sum of $p(x)$ over all possible values equals 1.

Example: rolling a fair six-sided die. The PMF is $p(x) = \frac{1}{6}$ for $x \in \{1, 2, 3, 4, 5, 6\}$ and $p(x) = 0$ otherwise.

Probability Density Functions

For a continuous random variable, the probability density function (PDF) $f(x)$ has a different interpretation than the PMF. The probability of landing in an interval $[a, b]$ is:

The PDF must satisfy $f(x) \geq 0$ for all $x$ and $\int_{-\infty}^{\infty} f(x) \, dx = 1$. Critically, $f(x)$ is not itself a probability, it can be greater than 1 in some regions. Only integrated over an interval does it produce a valid probability.

The Cumulative Distribution Function

Both discrete and continuous random variables have a cumulative distribution function (CDF):

The CDF gives the probability that the variable is less than or equal to some value. It is always non-decreasing and ranges from 0 to 1. For discrete variables, $F(x) = \sum_{t \leq x} p(t)$. For continuous variables, $F(x) = \int_{-\infty}^x f(t) \, dt$.

The CDF is useful because it works for any type of random variable and makes it easy to answer questions like “what is the probability the stock return is below -5%?”

Expected Value

The expected value (or mean) of a random variable is its long-run average, the center of gravity of its distribution.

For a discrete random variable:

For a continuous random variable:

Think of $E[X]$ as the balance point of the distribution. If you repeated the underlying random process infinitely many times and averaged the results, you would get $E[X]$.

Expected value has a linearity property that is extremely useful: $E[aX + b] = aE[X] + b$, regardless of whether $X$ and $a$ are independent.

Variance and Standard Deviation

The expected value tells you where the distribution is centered. The variance tells you how spread out it is:

where $\mu = E[X]$. Variance is the average squared distance from the mean. Because we square the distances, variance is always non-negative and is measured in squared units of the original variable, which is often inconvenient.

The standard deviation $\sigma = \sqrt{\text{Var}(X)}$ brings us back to the original units. For a normal distribution, roughly 68% of observations fall within one standard deviation of the mean, 95% within two, and 99.7% within three.

A Discrete Example: Number of Heads in Three Flips

Suppose you flip a fair coin three times and let $X$ be the number of heads. The sample space has 8 equally likely outcomes (HHH, HHT, HTH, THH, HTT, THT, TTH, TTT), and $X$ can take values 0, 1, 2, or 3.

The PMF is:

The expected value is $E[X] = 0 \cdot \frac{1}{8} + 1 \cdot \frac{3}{8} + 2 \cdot \frac{3}{8} + 3 \cdot \frac{1}{8} = \frac{12}{8} = 1.5$. On average, you expect one and a half heads in three flips, which makes sense, since each flip has a 50% chance of being heads and there are three flips, so $3 \times 0.5 = 1.5$.

A Continuous Example: Asset Returns

In quantitative finance, daily asset returns are modeled as continuous random variables. A common model is to assume a return $R \sim N(\mu, \sigma^2)$ normally distributed with mean $\mu$ and variance $\sigma^2$. We will study the normal distribution in detail in the next article.

Under this model, $P(R < -0.05)$ is the probability of a loss exceeding 5%. These are the kinds of risk questions that quant models are built to answer.

Simulating Random Variables in Python

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

np.random.seed(42)


mu = 0.0005      # Daily mean return (e.g., ~12% annual)
sigma = 0.02     # Daily volatility (e.g., ~32% annual)
n_days = 10_000

returns = np.random.normal(mu, sigma, n_days)

# Plot histogram of simulated returns
plt.figure(figsize=(10, 5))
plt.hist(returns, bins=50, density=True, alpha=0.6, label='Simulated returns')

# Overlay the theoretical normal PDF
x = np.linspace(mu - 4*sigma, mu + 4*sigma, 100)
plt.plot(x, stats.norm.pdf(x, mu, sigma), 'r-', label='Theoretical N(μ, σ²)')
plt.xlabel('Daily return')
plt.ylabel('Density')
plt.title('Simulated vs Theoretical Normal Distribution of Daily Returns')
plt.legend()
plt.grid(alpha=0.3)
plt.show()

# Calculate sample statistics
sample_mean = np.mean(returns)
sample_std = np.std(returns)
print(f"Sample mean: {sample_mean:.6f} (theoretical: {mu:.6f})")
print(f"Sample std:  {sample_std:.6f} (theoretical: {sigma:.6f})")

PROMPT

Use numpy, matplotlib, and scipy.stats to simulate 10,000 daily returns from a normal distribution, plot a histogram with the theoretical PDF, and compare sample vs theoretical stats.

Notice how the histogram of simulated returns closely follows the theoretical normal curve. This agreement improves with more simulations, a consequence of the law of large numbers, which we will explore later in this series.

References

Ross, S. M. A First Course in Probability, Chapters 2–3. Pearson. Papoulis, A. and Pillai, S. U. Probability, Random Variables, and Stochastic Processes, Chapter 5. McGraw-Hill.