Why MVO Weights Are Unstable (And What To Do About It)

Markowitz’s 1952 paper gives you a beautiful, elegant quadratic program. Solve it, you get the efficient frontier. Closed-form. Two-fund separation. Nobel-Prize-grade math.

There’s just one problem. In practice, it doesn’t work.

In 1989, Richard Michaud published a paper in the Financial Analysts Journal arguing that mean-variance optimization, as typically implemented, is an “error maximizer” rather than a “mean-variance optimizer.” He wasn’t being coy. He meant that the optimizer amplifies estimation errors in the inputs more than it captures real mean-variance structure in the outputs.

This article walks through why, with the math. Then we’ll look at three families of fixes: shrinkage, resampling, and robust optimization.

The Inputs Are Lies

Markowitz’s QP takes two inputs: $\mu$ and $\Sigma$. You estimate both from data. In the standard setup, you take historical returns, compute sample means and sample covariances, and feed them in.

This is where things go sideways.

Sample means of asset returns are spectacularly noisy. If you estimate expected return for a single stock from 60 months of monthly returns (5 years), the standard error of your estimate is the standard deviation of returns divided by $\sqrt{60}$. For a typical stock with 6% monthly vol, that’s $6\% / \sqrt{60} \approx 0.77\%$ per month, or roughly 9% annualized. The signal-to-noise ratio on a 10% expected return estimate is about 1. You’re guessing.

Covariance matrices are more stable, but they have their own issues. With $N$ assets and $T$ observations, you have $N(N+1)/2$ free parameters to estimate. For $N = 50$ and $T = 60$, that’s 1,275 parameters from 3,000 data points. The sample covariance matrix is well-known to be unstable when $T/N$ is small. Its eigenvalues are biased high in the trailing direction and biased low in the leading direction (this is the Marčenko-Pastur law).

You then feed these noisy estimates into an optimizer that takes the inverse of $\Sigma$ and computes $w^\star = \lambda \Sigma^{-1} \mu + \gamma \Sigma^{-1} \mathbf{1}$. The $\Sigma^{-1}$ operation amplifies estimation error in the small eigenvalues. The $\mu$ multiplication adds noise. You get $w^\star$ values that are wildly different from the “true” optimal weights, and worse, those weights are concentrated and unstable.

The Math of the Problem

Let’s be more precise. Suppose the true inputs are $(\mu_0, \Sigma_0)$, but you estimate $(\hat{\mu}, \hat{\Sigma})$. The optimizer returns $\hat{w}^\star(\hat{\mu}, \hat{\Sigma})$. The optimal portfolio under the true inputs is $w^\star(\mu_0, \Sigma_0)$.

A classic result from Kan & Zhou (2007), extending earlier work by Frost & Savarino (1986) and others, decomposes the expected loss from estimation error into four pieces:

Concretely, for the GMV portfolio, the expected squared tracking error is:

The thing in front, $\Sigma^{-1} \mathbf{1} \mathbf{1}^\top \Sigma^{-1}$, is what gets the error from your covariance estimate. If $\Sigma$ has small eigenvalues (concentrated risk), $\Sigma^{-1}$ blows up. If you estimated $\Sigma$ from too little data, $\text{Cov}(\hat{\Sigma})$ is large. Both multiply each other. That’s the “error maximizer” effect.

For the tangency portfolio it’s even worse, because you also have $\mu$ noise:

So the more confident you are in your return estimates, the more the optimizer will magnify any mistake you made. The optimal Markowitz portfolio is the least stable object you can construct from financial data.

What Michaud Actually Said

Here’s the punchline from Michaud (1989):

“Mean-variance optimization with sample estimates can be expected to produce an ‘optimized’ portfolio that is, with high probability, inferior to the equally-weighted portfolio in out-of-sample performance.”

This was a controversial thing to say in 1989. It implied that a sophisticated optimizer can lose to a 1/N heuristic on out-of-sample Sharpe. Two decades later, DeMiguel, Garlappi & Uppal (2009) showed exactly that, comparing 1/N against 14 optimized portfolios across 7 datasets. The optimized portfolios underperformed 1/N out-of-sample, often for decades.

This isn’t a Markowitz-bashing article. Markowitz’s theory is fine. The problem is that his theory assumes you know $(\mu, \Sigma)$. In practice you don’t, and the optimizer’s sensitivity to those inputs is brutal.

Fix 1: Shrinkage

The simplest, most robust fix: don’t use the sample estimates directly. Shrink them toward a structured target.

Ledoit & Wolf (2004) introduced a closed-form shrinkage estimator for the covariance matrix:

where $F$ is a “structured target”, typically a multiple of the identity, $F = \bar{\sigma}^2 I$, and $\delta \in [0, 1]$ is the shrinkage intensity, chosen analytically to minimize expected squared error:

In words: shrink the sample covariance toward the average variance times an identity, with intensity chosen so that the mean-squared-error of the shrunk estimator beats the sample estimator.

The Ledoit-Wolf shrinkage is “rotation invariant”, which means it doesn’t care which way your assets are oriented. It’s also the maximum-likelihood estimator under a Gaussian assumption with the prior that covariance should be close to $F$.

A more flexible version is “oracle approximating shrinkage” (OAS), which uses a different target and often performs better in high dimensions.

For $\mu$, you can shrink toward the cross-sectional mean $\bar{\mu} \mathbf{1}$, or toward zero, or toward an equilibrium prior (which is exactly what Black-Litterman does; see next article).

Effect of shrinkage: it pulls weights toward equal-weight or toward the structured target. This is mathematically guaranteed to reduce out-of-sample variance of the weights, at the cost of some in-sample bias. In practice, you almost always come out ahead.

Fix 2: Resampling

Michaud’s own proposed cure was resampling (later formalized in Michaud & Michaud, 1998, and operationalized in López de Prado’s Advances in Financial Machine Learning, 2018, Ch. 16).

The procedure:

1. Generate $B$ bootstrap samples from your historical returns.
2. For each bootstrap sample, estimate $(\hat{\mu}_b, \hat{\Sigma}_b)$ and compute the optimal portfolio $w_b^\star$.
3. Average the weights: $\bar{w} = \frac{1}{B} \sum_b w_b^\star$.

The intuition: bootstrap samples are noisy estimates of the same underlying distribution. Their optima are scattered around the true optimum. Averaging them gives you a more stable estimate of the central tendency. It also smooths out extreme weight allocations from any single bootstrap sample.

The math works because each $w_b^\star$ has the same expected value (the true optimum) and the bootstrap process provides independent (approximately) draws from the sampling distribution of the optimizer. The averaged weights have variance that scales like $1/B$ in the central limit regime.

In code, the core loop is short:

import numpy as np
from scipy.optimize import minimize

def resampled_weights(returns, target_return, n_bootstrap=200):
    n_obs, n_assets = returns.shape
    weights = np.zeros((n_bootstrap, n_assets))
    
    for b in range(n_bootstrap):
        # Bootstrap resample with replacement
        idx = np.random.choice(n_obs, size=n_obs, replace=True)
        boot = returns[idx]
        mu_b = boot.mean(axis=0)
        Sigma_b = np.cov(boot.T)
        
        # Solve MVO on this bootstrap sample
        weights[b] = solve_mvo(mu_b, Sigma_b, target_return)
    
    # Average the optimal weights
    return weights.mean(axis=0)

López de Prado’s implementation in mlfinlab wraps this with additional safety checks. He also points out an important subtlety: if you use shrinkage or robust inputs in step 2, you can compound the improvements. Resampling stacks with everything else.

The catch: resampling is slow. Each iteration is a QP solve. For $B = 200$ and $N = 100$ assets, you’re solving 200 quadratic programs. On a modern machine with cvxpy or quadprog, this is a few seconds. On a 1990s Bloomberg, this was infeasible, which is part of why Michaud’s ideas took 30 years to get adopted.

Fix 3: Robust Optimization

A third family of fixes treats the inputs as uncertain and optimizes the worst case. Goldfarb & Iyengar (2003) is the canonical reference. We’ll do a full article on it later in this module.

The idea: instead of solving

you solve

where $\mathcal{U}$ is an uncertainty set around $\hat{\Sigma}$. Common choices for $\mathcal{U}$:

• Box uncertainty: each $\sigma_{ij}$ can vary independently within $\pm \epsilon$.
• Ellipsoidal uncertainty: $\mathcal{U} = \{\Sigma : (\Sigma - \hat{\Sigma}) \in \mathcal{E}\}$ for some ellipsoid $\mathcal{E}$.

The robust counterpart is still a convex program (in fact, often still a QP or SOCP). It produces weights that are guaranteed to be near-optimal under any input in the uncertainty set. The price is that the resulting weights are typically more conservative than the nominal optimum.

Tütüncü & Koenig (2004) showed that robust MVO with ellipsoidal uncertainty is equivalent to MVO with a “robustified” covariance matrix:

i.e., adding a multiple of the identity. This is essentially Ledoit-Wolf shrinkage in disguise, but derived from a max-min rather than a min-MSE argument.

What to Use in Practice

Practical recommendations:

• Sample MVO: never use it for production. Educational only.
• Shrinkage (Ledoit-Wolf) on $\Sigma$: cheap, well-understood, almost always improves out-of-sample. Use it.
• Shrinkage on $\mu$: also good, but the choice of shrinkage target matters more. Black-Litterman gives you a principled target.
• Resampling: works well, but compute cost is non-trivial. López de Prado recommends it as a default for backtests.
• Robust MVO: useful when you have a strong prior on input uncertainty. Less commonly deployed in long-only equity portfolios.
• Constraints: this is the silent killer. Adding $\mathbf{1}^\top w = 1$, $w \geq 0$, and sector/asset bounds stabilizes weights enormously without any of the above machinery.

A reasonable production setup: Ledoit-Wolf shrinkage on $\Sigma$, Black-Litterman prior on $\mu$ (next article), resampling for final weights, with long-only and concentration constraints. That’s a defensible institutional setup.

What the Literature Says

• Michaud (1989). “The Markowitz Optimization Enigma: Is ‘Optimized’ Optimal?” Financial Analysts Journal 45(1): 31-42. The “error maximizer” critique.
• Michaud & Michaud (1998). Efficient Asset Management. Harvard Business School Press. The resampling proposal.
• Ledoit & Wolf (2004). “A Well-Conditioned Estimator for Large-Dimensional Covariance Matrices.” Journal of Multivariate Analysis 88(2): 365-411. The shrinkage estimator.
• Ledoit & Wolf (2003). “Improved Estimation of the Covariance Matrix of Stock Returns with an Application to Portfolio Selection.” Journal of Empirical Finance 10(5): 603-621.
• Frost & Savarino (1986). “An Empirical Bayes Approach to Efficient Portfolio Selection.” Journal of Financial and Quantitative Analysis 21(3): 293-305.
• Kan & Zhou (2007). “Optimal Portfolio Selection with Risk-Constraint.” Quantitative Finance 7(2): 165-185. The bias-variance decomposition.
• DeMiguel, Garlappi & Uppal (2009). “Optimal Versus Naive Diversification: How Inefficient is the 1/N Portfolio Strategy?” Review of Financial Studies 22(5): 1915-1953. The 1/N beating optimized portfolios result.
• Goldfarb & Iyengar (2003). “Robust Portfolio Selection Problems.” Mathematics of Operations Research 28(1): 1-21. Robust MVO foundations.
• López de Prado (2018). Advances in Financial Machine Learning. Wiley. Ch. 16 covers resampling in detail. ISBN: 978-1-119-48208-6.
• Tütüncü & Koenig (2004). “Robust Asset Allocation.” Annals of Operations Research 132(1-4): 157-187. Robust MVO equivalence.

A Simulation Study

The error-maximizer effect is most clearly seen in simulation. Here’s a complete experiment:

1. Generate $T = 60$ months of returns for $N = 50$ assets from a known true distribution (e.g., multivariate normal with $\mu_0, \Sigma_0$).
2. Compute sample estimates $\hat{\mu}, \hat{\Sigma}$.
3. Solve MVO. Compute the gap $\|w^\star(\hat{\mu}, \hat{\Sigma}) - w^\star(\mu_0, \Sigma_0)\|$.
4. Repeat 1000 times. Plot the histogram of gaps.

For $N = 50$, $T = 60$, the gap is often larger than $\|w^\star(\mu_0, \Sigma_0)\|$ itself. The estimated optimum is, in expectation, further from the truth than the equal-weight portfolio would be. This is the error-maximizer effect in action.

import numpy as np
from scipy.optimize import minimize

def simulate_mvo_error(N=50, T=60, n_trials=1000):
    # True parameters
    mu_0 = np.random.uniform(0.05, 0.15, N) / 12  # Monthly
    A = np.random.randn(N, N) * 0.02
    Sigma_0 = A @ A.T + np.eye(N) * 0.001  # PSD
    
    # True optimum (long-only for realism)
    def neg_sharpe_unconstrained(w):
        return -(w @ mu_0) / np.sqrt(w @ Sigma_0 @ w)
    
    # Run trials
    gaps_naive = []
    for _ in range(n_trials):
        # Generate sample
        returns = np.random.multivariate_normal(mu_0, Sigma_0, T)
        mu_hat = returns.mean(axis=0)
        Sigma_hat = np.cov(returns.T)
        
        # Solve MVO with sample estimates
        w_naive = np.linalg.solve(Sigma_hat, mu_hat)
        w_naive = w_naive / w_naive.sum()  # Normalize
        
        # True optimum (unconstrained, then normalized)
        w_true = np.linalg.solve(Sigma_0, mu_0)
        w_true = w_true / w_true.sum()
        
        gaps_naive.append(np.linalg.norm(w_naive - w_true))
    
    return np.mean(gaps_naive), np.std(gaps_naive)

mean_gap, std_gap = simulate_mvo_error()
print(f"Mean gap: {mean_gap:.4f}, Std: {std_gap:.4f}")

For typical parameters, mean gap is around 1.5-3.0 (in normalized weight space). The equal-weight portfolio has gap of about 0.5-1.0. So MVO with sample estimates is worse than equal-weight on average. This isn’t a fluke. It’s a structural feature.

The Bayesian Alternative

A more principled approach to estimation error: Bayesian inference on $(\mu, \Sigma)$.

The model:

with prior hyperparameters $(\mu_0, \Sigma_\mu, \Sigma)$.

The posterior:

The posterior mean is a shrinkage estimator, which pulls the sample mean $\bar{r}$ toward the prior mean $\mu_0$, with intensity depending on relative precisions.

The Bayesian optimal portfolio is:

which is the standard MVO with the posterior mean plugged in.

This is mathematically clean. It generalizes Black-Litterman (which is itself a Bayesian model with $\mu_0$ being the equilibrium return). It also generalizes James-Stein shrinkage (which is the empirical-Bayes version of this for $K = 1$).

Hierarchical Bayes

For multi-level estimation (e.g., sector portfolios containing stock portfolios), hierarchical Bayes gives you partial pooling. The idea:

Estimating the stock-level means requires data, but partial pooling toward the sector-level mean reduces estimation error. This is exactly the structure underlying factor models.

Brownian motion priors on $\mu_t$ for time-varying expected returns are an extension. The model:

with $r_t | \mu_t \sim \mathcal{N}(\mu_t, \Sigma)$. The Kalman filter gives the optimal $\mathbb{E}[\mu_t | r_{1:t}]$, which is a time-varying shrinkage estimator.

This is the right framework for active management with time-varying alpha. The Black-Litterman model can be extended to handle this case (Meucci, 2008).

Cross-Validation for Choosing Hyperparameters

Most of the methods above have hyperparameters: shrinkage intensity $\delta$, view covariance $\Omega$, robust uncertainty size. Choosing these via cross-validation is standard.

The procedure:

1. Split historical data into $K$ folds (time-series aware, not random).
2. For each fold, fit hyperparameters on the training portion.
3. Evaluate out-of-sample performance on the held-out portion.
4. Choose hyperparameters that maximize average held-out performance.

Combinatorial purged cross-validation (CPCV), from López de Prado, is the rigorous version for financial time series. It accounts for serial correlation and the multiple-testing problem.

In practice, most shops pick hyperparameters via backtest, not cross-validation. Backtests are easier to interpret and align with the way strategies are evaluated. The downside is overfitting: if you test 100 hyperparameter combinations, you’ll pick one that happened to work in the past.

The honest approach: split your data into train, validation, and test sets. Tune on validation. Final evaluation on test, only once. Treat the test set as a final exam you can take only once.

Why This Article Matters

If you’re going to run any MVO variant, you need to understand the estimation error problem. The math isn’t optional. The simulation results aren’t optional. The fixes aren’t optional.

The next 11 articles in this module are all, in some sense, responses to this one. Every method we’ll discuss (Black-Litterman, risk parity, HRP, CVaR, robust MVO) is a way of dealing with the fact that $(\mu, \Sigma)$ aren’t known.

What to Remember

Markowitz’s optimization is mathematically beautiful and practically unstable. The instability comes from estimation error in $\mu$ and $\Sigma$, amplified by the optimizer’s sensitivity to those estimates.

The error-maximizer effect is a structural feature of MVO, not a bug. Simulation shows MVO with sample estimates has higher average error than equal-weight. This is what Michaud was complaining about in 1989.

Three families of fixes exist: shrinkage (regularize the inputs), resampling (average over the optimizer’s instability), and robust optimization (optimize the worst case). Bayesian inference is a fourth approach that subsumes shrinkage. All four work.

The most defensible production setup combines them with sensible constraints: Ledoit-Wolf on $\Sigma$, Black-Litterman on $\mu$, resampling for final weights, long-only constraints throughout.

Next article: Black-Litterman. The Bayesian upgrade to MVO that turns $\mu$ estimation into a feature instead of a bug.