Hierarchical Risk Parity, the López de Prado Way

Mean-variance optimization has exactly one disease, and its name is $\Sigma^{-1}$. Inverting the covariance matrix amplifies the estimation error inside it; the smallest eigenvalues turn into the largest weights; and you end up holding a portfolio that is optimal on paper and disintegrates out of sample. That diagnosis has been in the literature since the 1980s, and the prescriptions have all run the same direction: shrink the matrix, constrain the weights, resample the inputs. Every one of them is an attempt to make the inverse behave.

Marcos López de Prado’s 2016 paper, “Building Diversified Portfolios that Outperform Out-of-Sample,” went the other way: don’t invert it. Hierarchical Risk Parity (HRP) allocates capital down a clustering tree, reading $\Sigma$ the whole time and never inverting it.

The cheapest way to understand HRP is not “it has three steps” but a single question that runs through all three: which part of the covariance matrix do you actually trust? HRP’s answer is unusually stingy, and each step trusts a different sliver. Once you can see what each step reads and what it deliberately ignores, both the strengths and the defects fall out.

The Fork in the Road

The setup is standard. You have $N$ assets, a covariance matrix $\hat{\Sigma}$ estimated from $T$ observations, and you want weights $w$.

MVO inverts $\hat{\Sigma}$ and solves a QP, which amounts to claiming that every entry of $\hat{\Sigma}$ is trustworthy enough to go through an inverse. Let $T/N$ get small and that claim is simply false. Risk parity backs off one step: it solves the equal-risk-contribution system, which needs no explicit inverse, but the solution still depends on the precise values of $\Sigma_{ij}$ and still requires iterative root-finding. HRP backs off furthest. Cluster the assets by correlation, allocate down the tree: stable, fast, no matrix inversion.

Its philosophy is that the structure of correlations tells you how to diversify even when the covariance matrix itself is noisy. You don’t need accurate $\Sigma_{ij}$ values. You need a roughly sensible ordering.

Step 1: Tree Clustering, Which Trusts Only the Ordering

First convert correlations into distances. The standard choice is

which is a genuine metric on correlations (it satisfies the triangle inequality), mapping $\rho_{ij} = 1$ to $d = 0$ and $\rho_{ij} = -1$ to $d = \sqrt{2}$. With a distance matrix in hand you run hierarchical clustering. HRP’s standard choice is single linkage, though other linkages run fine:

from scipy.cluster.hierarchy import linkage, dendrogram
from scipy.spatial.distance import squareform

# Correlation matrix
corr = np.corrcoef(returns.T)

# Distance matrix
dist = np.sqrt((1 - corr) / 2)

# Single-linkage clustering
dist_condensed = squareform(dist)
linkage_matrix = linkage(dist_condensed, method='single')

The output is a tree showing how assets merge as you lower the distance threshold: tightly correlated assets merge first, weakly correlated ones merge last. Notice how little this step extracts from $\hat{\Sigma}$. It extracts an ordering. Whether $\rho_{ij}$ is 0.62 or 0.68 barely matters as long as the relative magnitudes don’t flip. That is where HRP’s robustness comes from, and it is also the first thing worth being suspicious about: what guarantees the ordering is right?

The single-linkage choice is itself non-obvious. Single linkage is famous for “chaining”, meaning long, stringy clusters, while Ward linkage produces far more balanced trees. Empirically single linkage works well for HRP because it preserves the topology of the correlation matrix: strongly correlated groups stay stuck together and weakly correlated groups stay pushed apart, even in the presence of outliers. Ward linkage can split a correlated group when within-group variance is high, which is bad for HRP because it breaks up something that should be treated as one unit. Run it the other way, though, and if your correlation estimates are very noisy, average or complete linkage can be the more stable option, because single linkage is sensitive to noise in the distance metric. None of this is a theorem. It is empirical judgment, and the bill comes due later.

Step 2: Quasi-Diagonalization, Which Trusts Only the Blocks

The tree gives an ordering, and the second step permutes the covariance matrix along it so that similar assets sit next to each other. With $P$ the permutation induced by the tree:

This is not diagonalization in the linear-algebra sense. The matrix still has off-diagonal entries; they have merely been concentrated near the diagonal. That is what the “quasi” is doing in the name.

Why bother? The intuition comes back from Markowitz instability. When $\Sigma$ is diagonal-like, meaning the assets are nearly independent, the inverse is well-conditioned and weights are stable. When $\Sigma$ has large off-diagonal blocks, meaning assets clump into correlated groups, the inverse is ill-conditioned and weights thrash. Permuting doesn’t change the spectrum; it changes legibility. The high-correlation blocks float up onto the diagonal:

Original (random order):
Σ = | X X X . . . X |
    | X X X . . . X |
    | X X X . . . X |
    | . . . X X X . |
    | . . . X X X . |
    | . . . X X X . |
    | X X X . . . X |

Quasi-diagonalized (cluster order):
Σ_qd = | X X X X . . . |
        | X X X X . . . |
        | X X X X . . . |
        | X X X X . . . |
        | . . . X X X X |
        | . . . X X X X |
        | . . . X X X X |

Plot a heatmap of $\Sigma_{\text{qd}}$ and you see it: high correlation inside blocks, low correlation between them. That structure is the precondition for the next step. Recursive bisection makes one cut at a time, and each cut has to land somewhere weakly correlated, or else it is chopping a single unit in half and then pretending the halves are independent. The entire job of quasi-diagonalization is to make sure the knife lands where it should.

Step 3: Recursive Bisection, Which Trusts the Diagonal (Almost)

Only now does capital get allocated, and the procedure takes two sentences. All assets start as one cluster holding 100% of capital; split the cluster into two sub-clusters, divide capital between them by an inverse-variance split, then recurse into each sub-cluster until you hit leaf nodes. A leaf’s final weight is the product of every split coefficient along the path from root to leaf.

Everything hinges on the inverse-variance split. For two sub-clusters with variances $V_L$ and $V_R$:

The question is how $V_L$ is computed. A cluster holds a bundle of correlated assets, so its “variance” depends on what you assume about how it is held internally. There are at least four answers, and they read $\Sigma_L$ to very different depths.

The laziest treats the cluster as one asset held at unit exposure, $V_L = \mathbf{1}^\top \Sigma_L \mathbf{1}$, where $\Sigma_L$ is the submatrix of $\Sigma$ for cluster $L$. Slightly more disciplined is the equal-weight portfolio variance, $V_L = (\mathbf{1}/n_L)^\top \Sigma_L (\mathbf{1}/n_L) = \mathbf{1}^\top \Sigma_L \mathbf{1} / n_L^2$. At the far end sits the within-cluster minimum-variance portfolio, $V_L = 1 / (\mathbf{1}^\top \Sigma_L^{-1} \mathbf{1})$, which requires inverting a submatrix and thereby invites back in exactly the thing HRP set out to avoid.

López de Prado takes the third option: the inverse-variance portfolio (IVP) within the cluster, with weights

so each asset is weighted by the reciprocal of its own variance and normalized to sum to one. The cluster variance is the variance of that IVP held inside the cluster:

This deserves a pause. The weights $\tilde{w}$ use only the diagonal of $\Sigma_L$ and require no inversion, but the quadratic form $V_L$ uses the full $\Sigma_L$, off-diagonals included. So the line that “HRP uses correlations only for clustering, never for weighting” needs a caveat: within-cluster covariance does influence how much capital a cluster receives, just as a scalar rather than as an inverse. This is the one place HRP reads a number off the off-diagonal, and it is the extra diversification information it has over a plain IVP.

The split then reads

which is not 50/50. Whichever cluster has lower internal variance gets more capital, and “lower internal variance” bundles two distinct things: the cluster’s constituents are less volatile, and the cluster is already well diversified internally.

In practice the four choices give broadly similar answers. The third became standard because it needs no inversion, because it respects the relative volatilities inside the cluster, and because it is what López de Prado’s AFML implementation computes and what beat equal-weight in his out-of-sample tests. No theory says it is the right one.

In code:

def hrp_allocate(cov, order):
    n = len(cov)
    weights = np.ones(n)
    
    def _bisect(items):
        if len(items) <= 1:
            return
        # Split items into two halves
        mid = len(items) // 2
        left, right = items[:mid], items[mid:]

        # Compute cluster variances via the inverse-variance portfolio (IVP)
        cov_l = cov[np.ix_(left, left)]
        cov_r = cov[np.ix_(right, right)]
        w_l = 1 / np.diag(cov_l)
        w_l = w_l / w_l.sum()
        w_r = 1 / np.diag(cov_r)
        w_r = w_r / w_r.sum()
        v_l = w_l @ cov_l @ w_l
        v_r = w_r @ cov_r @ w_r

        # Inverse-variance split between clusters
        alpha_l = 1 / v_l / (1 / v_l + 1 / v_r)
        alpha_r = 1 - alpha_l

        # Update weights
        weights[left] *= alpha_l
        weights[right] *= alpha_r

        # Recurse
        _bisect(left)
        _bisect(right)
    
    _bisect(order)
    return weights / weights.sum()

Time $\mathcal{O}(N \log N)$, space $\mathcal{O}(N)$, against $\mathcal{O}(N^3)$ for the matrix inversion on the MVO side.

The Inverse-Variance Split Isn’t Improvised

That $\alpha_L \propto 1/V_L$ looks heuristic, but under one explicit assumption it is the optimum. Take two clusters L and R with covariance matrices $\Sigma_L$ and $\Sigma_R$, allocate $\alpha$ to L and $(1-\alpha)$ to R, and minimize the variance of the combination:

The block-diagonal form is the assumption: the two clusters are independent. Expanding,

Write $V_L = \mathbf{1}^\top \Sigma_L \mathbf{1}$ and $V_R = \mathbf{1}^\top \Sigma_R \mathbf{1}$, differentiate, set to zero:

The optimal split is proportional to the other cluster’s variance, which is the same as being proportional to the reciprocal of your own. That is the inverse-variance split.

The derivation holds each cluster at equal weights inside, via that $\mathbf{1}$ vector, but swapping in HRP’s actual within-cluster IVP changes nothing: substitute $\tilde{w}^\top \Sigma_L \tilde{w}$ for $\mathbf{1}^\top \Sigma_L \mathbf{1}$ and the formula stands. The split rule is invariant to the choice of within-cluster weights, since any scalar $V_L$ summarizing the cluster’s risk yields the same $\alpha_L \propto 1/V_L$, and the recursion handles within-cluster allocation at later steps regardless.

Which puts quasi-diagonalization in a sharper light. It isn’t cosmetic. It exists to manufacture the conditions under which “the two clusters are independent” is roughly true. Land the cut in a weakly correlated place, the block-diagonal form approximately holds, and the derivation approximately applies. The chain is self-consistent.

What the Paper Actually Shows

The 2016 paper claims three properties for HRP. Two are visible from the algorithm alone: no matrix inversion, since HRP works with the covariance matrix directly and thereby sidesteps the eigenvalue amplification that plagues MVO; and hierarchical diversification, since capital flows top-down through the cluster structure and you get diversification both across clusters and within them. The third, out-of-sample performance, is an empirical claim and deserves separate treatment.

The evidence in the paper is a Monte Carlo experiment, not a cross-section of real market datasets. Returns are simulated with a block correlation structure, portfolios are built on an in-sample covariance estimate, and their behavior is measured out of sample against CLA (the mean-variance line) and IVP. HRP’s out-of-sample variance comes in materially below both, by a margin measured in tens of percent rather than a few percentage points. Go to the original for the exact figures rather than trusting a secondhand retelling.

The most instructive corner of that design is the large-$N$, small-$T$ end. Picture 500 stocks and 60 months: Markowitz has to invert a 500×500 matrix estimated from 60 observations, so instability isn’t a risk, it’s a certainty. HRP simply never does that, and the gap opens. Run it the other way and the property is just as informative: if the simulated returns have no cluster structure at all, the tree is arbitrary and recursive bisection degenerates to something close to equal weight. HRP converges to 1/N rather than beating it. That is a healthy failure mode. It does not pretend to find structure that isn’t there.

Where It Parts Company With Risk Parity

Risk parity allocates by equal risk contribution; HRP allocates by inverse cluster variance. Related, but they swallow different amounts of information.

Risk parity needs a fully specified covariance matrix, solves a system of nonlinear equations, and depends on the precise values of $\Sigma_{ij}$. HRP needs the structure of the correlation matrix, meaning the tree topology, with off-diagonal correlations entering only through the clustering and through the within-cluster $V_L$, never directly into a weight formula.

The consequence is predictable. Perturb $\hat{\Sigma}$ slightly and HRP’s weights move slightly, while risk parity’s weights can move a lot. Empirically HRP turns over less than risk parity, because the tree is stable under small perturbations to $\Sigma$ while risk parity re-optimizes continuously. After transaction costs, that difference is often worth more than the out-of-sample Sharpe gap.

Network Theory Is the Same Thing Said Differently

Correlation-based hierarchical clustering sits very close to network theory. Read the correlation matrix as a weighted graph whose edges are pairwise correlations, and the minimum spanning tree (MST) connects all assets with $N-1$ edges at minimum total weight, handing you a hierarchy directly. Under the $\sqrt{(1-\rho)/2}$ family of distance metrics, single-linkage clustering and the MST correspond. HRP runs standard hierarchical clustering rather than an MST, but the topology they read is the same one.

That connection is why HRP works on financial networks: correlations genuinely have cluster structure (sectors, factors, regions) and the tree captures it. It also marks the boundary of usefulness. If your asset set has no such block structure, the tree is fitting noise.

Its Three Defects

The single-linkage choice is arbitrary. Different linkage methods give different trees, and different trees give different weights. Single linkage works well empirically, but nothing in theory says it should dominate Ward, complete, or average linkage. Raffinot (2018) showed the choice of linkage has a significant effect on out-of-sample performance. Back to the opening question: HRP stakes everything on the ordering, and the algorithm producing that ordering was picked by hand.

HRP doesn’t use expected returns. It is a pure risk-based allocation, so a strong view that one asset will beat another has nowhere to go. You can hack around it by adding a synthetic “view asset” with the desired correlation structure, but it’s awkward; the cleaner route is to compute the tree and quasi-diagonalization as usual and then tilt the final weights with Black-Litterman posterior returns.

The inverse-variance split is heuristic. The derivation above holds only under independence between clusters, and quasi-diagonalization can only make that assumption approximately true. Nothing proves inverse cluster variance is the optimal split; it is a reasonable choice motivated by single-asset mean-variance, not a result from first principles. A more principled route splits by equal risk contribution at the cluster level, giving a different set of weights. Raffinot’s follow-on work does exactly that, under the name Hierarchical Equal Risk Contribution (HERC): every level of the recursion uses an ERC split instead of an inverse-variance one. HERC is theoretically cleaner and slower, since each split requires solving an ERC system, and empirically the two land in similar out-of-sample territory.

Two other variants are worth knowing. Solving a small MVO inside each cluster instead of equal-weighting adds risk-based optimization back at the cluster level; adding long-only bounds, position caps, and sector limits during the recursion makes the math messier but the output compatible with real mandates. Both are defensible extensions.

Defects and all, HRP is a serious tool. Several quant firms have adopted it, and mlfinlab carries a clean implementation. For portfolios with $N > T$, or with notoriously noisy covariance estimates, it is a defensible default.

When It Is Worth Reaching For

The rough boundaries: below about 20 assets, any method works and equal weight is fine, so don’t bother with HRP. Between 20 and 100, HRP and Ledoit-Wolf-shrunk MVO both work, and you should try both and see what fits. Above 100, HRP starts to dominate, because the covariance matrix is too noisy for MVO, and while equal weight remains usable it gives you no diversification structure at all.

One trap is easy to walk into. If volatilities differ sharply across assets, say a mix of equities and bonds, HRP does not hand you “equal risk contribution”; it hands you “equal cluster variance.” If you want true ERC you need a post-processing step that adjusts weights to satisfy the constraint. A common hybrid gives HRP the cluster-level allocation and runs a constrained ERC inside each cluster, taking the topological robustness from one and the diversification property from the other.

References

• López de Prado (2016). “Building Diversified Portfolios that Outperform Out-of-Sample.” Journal of Portfolio Management 42(4): 59-69. The HRP paper.
• López de Prado (2018). Advances in Financial Machine Learning. Wiley. Ch. 16 has the implementation details. ISBN: 978-1-119-48208-6.
• Raffinot (2018). “Hierarchical Clustering-Based Asset Allocation.” Journal of Portfolio Management 44(2): 89-99. Critique and extensions; HERC comes from the author’s companion work.
• Tola, Lillo, Gallegati & Mantegna (2008). “Cluster Analysis for Portfolio Optimization.” Journal of Economic Dynamics and Control 32(1): 235-258. Earlier work on clustering-based allocation.
• DeMiguel, Garlappi & Uppal (2009). “Optimal Versus Naive Diversification: How Inefficient Is the 1/N Portfolio Strategy?” Review of Financial Studies 22(5): 1915-1953. Why the equal-weight benchmark is so hard to beat.
• Bruder, Gaussel, Richard & Roncalli (2013). “Regularization of Portfolio Allocation.” SSRN Working Paper. Comparison of risk-based allocation methods.

Next article: minimum-variance portfolio. The simplest risk-based MVO, and why it’s more robust than expected-return-based MVO.