Covariance and Correlation Matrices

Covariance: Measuring How Variables Move Together

Given two random variables $X$ and $Y$ with means $\mu_X$ and $\mu_Y$, the covariance measures whether they tend to move in the same direction:

When $X$ is above its mean and $Y$ is also above its mean, the product $(X - \mu_X)(Y - \mu_Y)$ is positive and covariance increases. When one is above its mean and the other is below, the product is negative and covariance decreases. If knowing $X$‘s position tells you nothing about $Y$‘s position, the positive and negative products cancel and covariance is near zero.

A limitation of covariance is that it depends on the units of $X$ and $Y$. If $X$ is measured in dollars and $Y$ in basis points, the magnitude of the covariance reflects those arbitrary scales.

Correlation: Normalized Covariance

Correlation fixes this by normalizing covariance to a scale that runs from -1 to +1:

A correlation of $+1$ means perfect positive co-movement, $X$ and $Y$ are perfectly aligned. A correlation of $-1$ means perfect opposite movement. A correlation near $0$ means no linear relationship.

Because correlation is scale-free, it is easier to interpret than covariance. A correlation of $0.7$ between two assets tells you they move together substantially, regardless of whether you measure returns in percent or basis points.

The Covariance Matrix

When you have $n$ assets, the covariance matrix $\mathbf{\Sigma}$ organizes all variances and covariances in one structure:

The diagonal entries are the variances of individual assets. The off-diagonal entries are the covariances between pairs of assets.

The correlation matrix is the normalized version, every diagonal entry is 1 (an asset is perfectly correlated with itself), and every off-diagonal entry is the correlation between a pair of assets:

Required Properties: Symmetry and Positive Semi-Definiteness

A valid covariance matrix has two non-negotiable properties.

Symmetry: $\Sigma_{ij} = \Sigma_{ji}$. Covariance is symmetric by definition, the covariance between asset $i$ and asset $j$ is the same as between $j$ and $i$.

Positive semi-definiteness: all eigenvalues of $\mathbf{\Sigma}$ are greater than or equal to zero. A covariance matrix with a negative eigenvalue would imply a portfolio with negative variance, an impossibility. You can verify this in Python:

import numpy as np

Sigma = np.array([[1.0, 0.5, 0.2],
                  [0.5, 1.0, 0.3],
                  [0.2, 0.3, 1.0]])

eigenvalues = np.linalg.eigvalsh(Sigma)
print("Eigenvalues:", eigenvalues)
print("All non-negative:", np.all(eigenvalues >= 0))

PROMPT

Use numpy to check whether a 3x3 symmetric matrix is positive definite by looking at its eigenvalues.

A covariance matrix constructed as $\mathbf{\Sigma} = \frac{1}{T-1}\mathbf{X}^\top \mathbf{X}$ is positive semi-definite by construction, because for any vector $\mathbf{w}$,

This holds whether or not you subtract column means: centering turns $\mathbf{X}$ into $\mathbf{X}_c$, but $\mathbf{X}_c^\top \mathbf{X}_c = \mathbf{X}^\top \mathbf{X} - T\bar{x}\bar{x}^\top \succeq \mathbf{X}^\top \mathbf{X}$ is still PSD by the same argument (it is a sum of two PSD matrices). Centering affects the values of the eigenvalues and the interpretation of the diagonal (variance vs. second moment), but not the PSD property. Estimation noise, sample size, and choice of divisor also do not invalidate PSD, the bound $\mathbf{X}^\top\mathbf{X}/(T-1) \succeq 0$ is purely algebraic.

Therefore, if you actually compute $\mathbf{X}^\top\mathbf{X}/(T-1)$ and observe a negative eigenvalue, the cause is not statistical; it is numerical: floating-point round-off accumulating across rows, or an error in the construction itself (you built something that is not actually of the form $\mathbf{X}^\top\mathbf{X}/(T-1)$, e.g., you mixed in uncentered cross-products, used a different denominator, or substituted an invalid correlation/covariance estimate that doesn’t have this algebraic form).

Computing in Python

import numpy as np
import pandas as pd


np.random.seed(42)
returns = pd.DataFrame({
    "AAPL": np.random.randn(252) * 0.02 + 0.001,
    "MSFT": np.random.randn(252) * 0.018 + 0.0008,
    "GOOG": np.random.randn(252) * 0.025 + 0.0012,
    "AMZN": np.random.randn(252) * 0.022 + 0.001
})

# Covariance matrix (annualized: daily var * 252)
cov_matrix = returns.cov() * 252
print("Annualized Covariance Matrix:\n", cov_matrix)

# Correlation matrix
corr_matrix = returns.corr()
print("\nCorrelation Matrix:\n", corr_matrix)

# Manual check
std = returns.std() * np.sqrt(252)
manual_corr = cov_matrix / np.outer(std, std)
print("\nManually computed correlation (should match):\n", manual_corr)

PROMPT

Use numpy and pandas to simulate daily returns for a few assets, then compute the annualized covariance and correlation matrices and cross-check the correlation.

Portfolio Variance Through Matrix Algebra

Modern portfolio theory expresses portfolio variance compactly using matrices. Given a vector of portfolio weights $\mathbf{w}$ and the covariance matrix $\mathbf{\Sigma}$:

This scalar output is the weighted sum of all pairwise covariances, where each weight product $w_i w_j$ multiplies the covariance between assets $i$ and $j$.

import numpy as np

# Four assets, covariance matrix (annualized)
Sigma = np.array([[0.0400, 0.0180, 0.0120, 0.0150],
                  [0.0180, 0.0324, 0.0140, 0.0160],
                  [0.0120, 0.0140, 0.0625, 0.0200],
                  [0.0150, 0.0160, 0.0200, 0.0484]])

# Equal weights
w = np.array([0.25, 0.25, 0.25, 0.25])

# Portfolio variance
port_var = w @ Sigma @ w
port_std = np.sqrt(port_var)
print(f"Portfolio variance: {port_var:.6f}")
print(f"Portfolio std dev: {port_std:.4f} ({port_std*100:.2f}%)")

PROMPT

Use numpy to compute the variance and standard deviation of an equal-weight portfolio from a covariance matrix.

The Small-$n$, Large-$p$ Problem

A serious practical issue arises when the number of assets $n$ is larger than the number of observations $T$. The sample covariance matrix $\mathbf{S} = \frac{1}{T-1}\mathbf{R}_c^\top\mathbf{R}_c$ (where $\mathbf{R}_c$ is the centered return matrix, with each column’s mean subtracted from every entry) is then singular, it has no inverse and eigenvalues include zeros.

With 500 assets and only 252 days of data, the sample covariance matrix is essentially unreliable. Many regularization techniques exist to address this: shrinking toward a structured target (like a constant-correlation or single-factor model), factor-based covariance estimation, or using random matrix theory to filter noise.

This is one reason PCA matters, by reducing $n$ assets to $k$ principal components, you dramatically reduce the number of parameters to estimate and sidestep singularity.

References

Ruppert, Statistics and Data Analysis for Financial Engineering, 2nd edition, Springer, Chapter 3. ISBN: 978-1493926134.

Markowitz, “Portfolio Selection,” The Journal of Finance, 7(1): 77–91, 1952.