Principal Component Analysis

The Goal: Finding Directions That Matter

When you have 500 assets, a 500 × 500 symmetric covariance matrix contains $\frac{500 \times 501}{2} = 125{,}250$ unique entries (the diagonal plus the upper or lower triangle). Most of that matrix is noise, structure that does not reflect genuine relationships. Principal Component Analysis (PCA) finds the few directions in your data that actually contain the signal, discarding the rest.

The core idea is straightforward: in any dataset, some directions vary more than others. The direction of maximum variance is the most informative direction, it captures the most structure with the fewest dimensions. PCA finds all the orthogonal directions of decreasing variance, then lets you keep only the top $k$ of them.

“Orthogonal” means the directions are uncorrelated with each other, they point in independent directions in the data’s feature space.

The Five Steps of PCA

Given a data matrix $\mathbf{X}$ with $n$ observations (rows) and $p$ variables (columns):

Step 1: Center the data. Subtract the mean of each column so every variable has mean zero. This ensures the first principal component is not simply pointing toward the data’s average location.

Step 2: Compute the covariance matrix. For centered data, this is simply:

Step 3: Find eigenvectors and eigenvalues of $\mathbf{\Sigma}$. Each eigenvector is a principal component direction, and its eigenvalue tells you how much variance lies along that direction.

Step 4: Sort by eigenvalue (largest first). The eigenvector with the largest eigenvalue points in the direction of greatest variance. The second-largest eigenvalue gives the next orthogonal direction, and so on.

Step 5: Choose $k$ components and project the data. The explained variance ratio for component $i$ is:

Doing It in Python

import numpy as np
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt


np.random.seed(42)
n_assets = 20
n_days = 500

# Generate correlated returns using Cholesky decomposition
L = np.random.randn(n_assets, n_assets) * 0.3
Sigma = L @ L.T + np.eye(n_assets) * 0.5  # positive definite
returns = np.random.randn(n_days, n_assets) @ np.linalg.cholesky(Sigma).T

# Manual PCA
returns_centered = returns - returns.mean(axis=0)
cov_matrix = np.cov(returns_centered.T)
eigenvalues, eigenvectors = np.linalg.eigh(cov_matrix)
eigenvalues = eigenvalues[::-1]  # descending

# Variance explained
total_var = eigenvalues.sum()
explained = eigenvalues / total_var
cumulative = np.cumsum(explained)

# Plot explained variance
plt.figure(figsize=(10, 5))
plt.bar(range(1, n_assets + 1), explained * 100, alpha=0.6, label="Individual")
plt.plot(range(1, n_assets + 1), cumulative * 100, "ro-", label="Cumulative")
plt.axhline(y=95, color="g", linestyle="--", label="95% threshold")
plt.xlabel("Principal Component")
plt.ylabel("Variance Explained (%)")
plt.title("PCA: Variance Explained by Component")
plt.legend()
plt.show()

# How many components to reach 95%?
k_95 = np.argmax(cumulative >= 0.95) + 1
print(f"Components to explain 95% of variance: {k_95}")

PROMPT

Use numpy and matplotlib to simulate correlated returns, do PCA manually via eigendecomposition of the covariance matrix, and plot the variance explained.

Using sklearn makes this even more compact:

pca = PCA(n_components=0.95)  # keep 95% of variance
reduced = pca.fit_transform(returns)

print(f"Original shape: {returns.shape}")
print(f"Reduced shape:  {reduced.shape}")
print(f"Components used: {pca.n_components_}")

PROMPT

Use scikit-learn’s PCA to reduce a returns matrix keeping 95% of the variance and print the shapes and the number of components.

Financial Application: Reducing a 500-Stock Correlation Matrix

The classic application in quant finance is reducing a massive correlation matrix to its essential factors. Instead of tracking how each of 500 stocks moves relative to each of the other 499 stocks, PCA might reveal that just 5 principal components explain 80% of the variation, and those components correspond to interpretable market factors like market beta, size, value, momentum, and volatility.

This is not a coincidence. Factor models in economics and finance were developed partly because the same intuition, that most variation in financial data can be explained by a small number of latent factors, predates PCA but is mathematically equivalent to it.

import numpy as np

# Simulated 100-asset monthly returns
np.random.seed(0)
returns = np.random.randn(120, 100) * 0.02

# Correlation matrix
corr = np.corrcoef(returns.T)

# PCA on correlation matrix
eigenvalues, _ = np.linalg.eigh(corr)
eigenvalues = eigenvalues[::-1]

# Explained variance
var_explained = eigenvalues / eigenvalues.sum()
cum_var = np.cumsum(var_explained)

# How many components explain 50%, 70%, 90%?
for threshold in [0.50, 0.70, 0.90]:
    k = np.argmax(cum_var >= threshold) + 1
    print(f"{threshold*100:.0f}% variance: {k} components")

PROMPT

Use numpy to run PCA on the correlation matrix of 100 simulated assets and find how many components reach 50, 70, and 90 percent of variance.

Overfitting: The Risk of Too Many or Too Few Components

PCA is a tool, and like any tool it can be misused. Keeping too many components, close to $n$ retains noise along with signal. The sample covariance matrix is itself a noisy estimate of the true covariance, and eigendecomposing it does not remove that noise. In fact, the smallest eigenvalues of the sample covariance matrix are mostly noise.

Keeping too few components discards real structure. If the true data-generating process has 20 meaningful factors but you keep only 3, you are underfitting, losing genuine relationships that affect returns.

A practical rule: look for an “elbow” in the explained variance curve, the point where adding more components yields diminishing returns. Alternatively, keep enough components to explain 70–90% of variance, depending on context.

References

Jolliffe, Principal Component Analysis, 2nd edition, Springer, Chapters 1–2. ISBN: 978-0387954424.

Hastie, Tibshirani & Friedman, The Elements of Statistical Learning, 2nd edition, Springer, Chapter 3. ISBN: 978-0387848570.