Eigenvalues and Eigenvectors

What an Eigenvector Is

When a matrix $\mathbf{A}$ multiplies a vector $\mathbf{v}$, it transforms that vector, rotating it, stretching it, compressing it. An eigenvector is a special vector that does not get rotated by the transformation. It only gets scaled.

Here $\mathbf{v}$ is the eigenvector and $\lambda$ is the eigenvalue: the factor by which the eigenvector is stretched or compressed. If $\lambda = 2$, the eigenvector doubles in length. If $\lambda = -0.5$, it flips direction and halves in length.

The word “eigen” comes from German and means “own” or “characteristic.” An eigenvector is the matrix’s “own” direction, intrinsic to the matrix, not arbitrary.

Finding Eigenvalues

The equation $\mathbf{A}\mathbf{v} = \lambda \mathbf{v}$ can be rewritten as:

For a non-zero $\mathbf{v}$ to satisfy this, the matrix $(\mathbf{A} - \lambda\mathbf{I})$ must be singular, its determinant must be zero. This gives the characteristic equation:

Solving this equation for $\lambda$ gives the eigenvalues. Each eigenvalue is then substituted back to find its corresponding eigenvector.

For a $2 \times 2$ matrix, the characteristic equation is a quadratic, you can solve it by hand. For larger matrices, numerical software is essential.

Eigendecomposition

Once you have all $n$ eigenvalues and eigenvectors of an $n \times n$ matrix $\mathbf{A}$, you can write:

where $\mathbf{Q}$ is the matrix whose columns are the eigenvectors, and $\mathbf{\Lambda}$ is a diagonal matrix with the eigenvalues on the diagonal:

This is the eigendecomposition of $\mathbf{A}$.

For symmetric matrices: and covariance matrices are always symmetric, the eigenvectors are orthogonal to each other, which means $\mathbf{Q}^{-1} = \mathbf{Q}^\top$. This simplifies the decomposition significantly:

Properties of Eigenvalues

Several relationships connect eigenvalues to the matrix’s other properties:

The trace (sum of diagonal elements) equals the sum of eigenvalues:

The determinant equals the product of eigenvalues:

Symmetric matrices always have real eigenvalues. Complex eigenvalues would require oscillatory behavior, rotating vectors rather than just scaling them. A covariance matrix, which is symmetric by definition, never rotates vectors. Its eigenvalues are always real numbers.

Positive definite matrices have all eigenvalues greater than zero. Covariance matrices are positive semi-definite (all eigenvalues $\geq 0$) and are positive definite if no asset is a perfect linear combination of others.

Computing in Python

import numpy as np


Sigma = np.array([[1.0, 0.5, 0.3],
                  [0.5, 1.0, 0.4],
                  [0.3, 0.4, 1.0]])

# For symmetric matrices, use eigh (numerically more stable)
eigenvalues, eigenvectors = np.linalg.eigh(Sigma)

# Eigenvalues are sorted in ascending order
print("Eigenvalues:", eigenvalues)
print("Eigenvectors (columns):\n", eigenvectors)

# Verify: A @ v = lambda * v
for i in range(3):
    lam = eigenvalues[i]
    v = eigenvectors[:, i]
    lhs = Sigma @ v
    rhs = lam * v
    print(f"Eigenvalue {i}: {np.allclose(lhs, rhs)}")

PROMPT

Use numpy’s np.linalg.eigh to find the eigenvalues and eigenvectors of a symmetric covariance matrix and verify each eigenpair.

For non-symmetric matrices, use np.linalg.eig instead of np.linalg.eigh. The non-symmetric version may return complex eigenvalues, which you would handle differently.

Application: PCA and Variance Decomposition

In the article on principal component analysis (PCA), you will see eigenvalues put to immediate use. Given a covariance matrix of asset returns, the eigenvalues tell you how much total variance each principal component captures.

Because a covariance matrix is symmetric and positive semi-definite:

• Each eigenvalue $\lambda_i \geq 0$
• The sum of all eigenvalues equals the total variance (trace of $\Sigma$)
• The fraction $\lambda_i / \sum_j \lambda_j$ is the proportion of total variance explained by component $i$

In practice, a 500-asset covariance matrix might have 95% of its variance concentrated in the top 20 eigenvalues. The remaining 480 eigenvalues are near zero, noise that does not meaningfully describe how the portfolio moves. This is how PCA achieves dimensionality reduction.

import numpy as np

# Simulated 5-asset covariance matrix
Sigma = np.array([[1.00, 0.80, 0.40, 0.20, 0.10],
                  [0.80, 1.00, 0.50, 0.30, 0.15],
                  [0.40, 0.50, 1.00, 0.60, 0.25],
                  [0.20, 0.30, 0.60, 1.00, 0.70],
                  [0.10, 0.15, 0.25, 0.70, 1.00]])

eigenvalues, _ = np.linalg.eigh(Sigma)

# Sort descending (eigh returns ascending)
eigenvalues = eigenvalues[::-1]

# Variance explained by each component
total_var = eigenvalues.sum()
var_explained = eigenvalues / total_var
cum_var = np.cumsum(var_explained)

print("Component | Eigenvalue | Variance % | Cumulative %")
for i in range(5):
    print(f"    PC{i+1}   |  {eigenvalues[i]:.4f}   |   {var_explained[i]*100:5.1f}   |    {cum_var[i]*100:5.1f}")

PROMPT

Use numpy to compute the eigenvalues of a covariance matrix, sort them, and print the variance explained per component.

References

Strang, Introduction to Linear Algebra, 5th edition, Wellesley-Cambridge Press, Chapter 6. ISBN: 978-0980232776.

Lax, Linear Algebra and Its Applications, 2nd edition, Wiley, Chapter 7. ISBN: 978-0471751564.