CVaR Optimization, the Linear Programming Way

Rockafellar & Uryasev (2000) rewrote a quantity that seems to need the quantile first as a convex minimization, and tail-risk optimization collapsed into a linear program. Plus why the hard part was never the solver.

2 of 12 ~15 min

Variance is a fine risk measure for a Gaussian world. We are not in a Gaussian world. Equity returns have fat tails, bond return distributions are asymmetric, and crypto is essentially unbounded on the downside. Variance misses all of it.

Conditional Value-at-Risk (CVaR), also called Expected Shortfall or Expected Tail Loss, or read plainly as “the average loss in the worst α% of cases”, is the institutional answer. It is a coherent risk measure and it looks at the part of the distribution where losses actually live. But what turned it from a nice definition into a usable tool was Rockafellar and Uryasev’s 2000 paper: they showed that this quantity, which appears to require knowing a quantile before you can compute it, can be written as a convex minimization. Plug in historical scenarios and the whole thing collapses into a linear program.

What follows is the definition, the case against variance, and then the LP built up piece by piece. Once it is built you will see that the hard part is not on the solver’s side of the problem.

What Variance Gets Wrong

Variance punishes upside and downside equally. A portfolio with 30% upside vol and 0% downside vol has the same variance as one with 0% upside and 30% downside, and to anyone whose money is in it those are not remotely the same object.

Worse, it is a quadratic measure and cannot see the shape of the tail. A portfolio with low variance but a 5% chance of losing 50% can carry the same “variance” as one with no tail risk and moderate day-to-day swings. Real investors care about that 5%.

Kritzman, Page and Turkington documented the point in “Regime Shifts: Implications for Dynamic Strategies” (Financial Analysts Journal, 2012): volatility differs far less across regimes than tail risk does. Proxy risk with volatility and two regimes look similar. Measure the tail and they are not the same animal.

CVaR fixes both problems at once. It is asymmetric, looking only at losses, and it is a tail measure, looking only at the worst α% of scenarios.

The Definition, and a Subscript People Write Backwards

Let $X$ be portfolio loss, so higher $X$ is worse, let $F_X(x) = P(X \leq x)$ be its CDF, and let $\alpha \in (0, 1)$ be the tail probability ( $\alpha = 0.05$ for the worst 5%).

Value-at-Risk at level $\alpha$ is the threshold

\text{VaR}_\alpha(X) = \min \{ x : F_X(x) \geq 1 - \alpha \}

Note the $1 - \alpha$ on the right, not $\alpha$ . Because $X$ is a loss, “the worst 5%” sits in the upper tail, so what you want is the 95th percentile. Flipping this subscript is one of the most common slips in the literature, and once it is flipped every $1/\alpha$ coefficient downstream stops meaning anything.

CVaR, the Expected Shortfall at level $\alpha$ , is the conditional expectation of $X$ given that $X$ exceeds VaR:

\text{CVaR}_\alpha(X) = \mathbb{E}[X \mid X \geq \text{VaR}_\alpha(X)]

Take the worst α% of cases, average the losses. That is all it is.

For continuous distributions there is a clean integral representation:

\text{CVaR}_\alpha(X) = \frac{1}{\alpha} \int_0^\alpha \text{VaR}_u(X) du

Sweep the tail probability from 0 to $\alpha$ and average every VaR along the way. For $\alpha = 0.05$ , that is the average of the worst 5% of outcomes.

Coherence: Four Axioms and Two Failures

A coherent risk measure (Artzner et al., 1999) satisfies four axioms. Monotonicity: if $X \leq Y$ in every state, then $\rho(X) \leq \rho(Y)$ . Translation invariance: $\rho(X + c) = \rho(X) + c$ for any constant $c$ . Subadditivity: $\rho(X + Y) \leq \rho(X) + \rho(Y)$ , meaning diversification must not increase risk. Positive homogeneity: $\rho(\lambda X) = \lambda \rho(X)$ for $\lambda \geq 0$ .

Variance fails on the first two immediately: it does not measure loss at all, and adding a constant to every outcome leaves it untouched. VaR fails on the third, a famous result. Two portfolios with VaR of $10 each can combine into a portfolio whose VaR exceeds $20. This is not a pathological construction; two default exposures with non-overlapping tails will do it. A risk number that can grow because you diversified is not something you can set limits against.

CVaR passes all four. That is why it became the mathematically defensible choice for institutional risk management.

The Rockafellar-Uryasev Formulation

The definition above has a practical problem. To compute CVaR you first need VaR, and VaR is a quantile, which as a function of the weights $w$ is neither continuous nor convex. Feeding it straight to an optimizer is a disaster.

Rockafellar and Uryasev’s move was to turn the quantile from something you must solve for first into something that falls out of the solution:

\text{CVaR}_\alpha(X) = \min_{\zeta} \left\{ \zeta + \frac{1}{\alpha} \mathbb{E}[(X - \zeta)^+] \right\}

where $(X - \zeta)^+ = \max(X - \zeta, 0)$ and $\zeta$ is a free variable. Minimize over $\zeta$ and the optimal $\zeta^\star$ turns out to equal $\text{VaR}_\alpha(X)$ , while the optimal objective value equals $\text{CVaR}_\alpha(X)$ . You never have to know the quantile in advance; the optimization hands it to you.

The result matters because $\mathbb{E}[(X - \zeta)^+]$ is a linear functional of the distribution of $X$ , and $\max(\cdot, 0)$ is piecewise linear. Convexity and linear structure, both in one line.

Building the LP

Connect it to a portfolio. Let $w$ be the weights, $r_j$ the return of asset $j$ , and $L_j = -r_j$ the loss, so the portfolio loss is

X(w) = -w^\top r = \sum_j w_j L_j

Minimize CVaR subject to a target return:

\begin{aligned} \min_{w, \zeta} \quad & \zeta + \frac{1}{\alpha T} \sum_{t=1}^T u_t \\ \text{s.t.} \quad & u_t \geq -w^\top r^{(t)} - \zeta, \quad \forall t \\ & u_t \geq 0, \quad \forall t \\ & \mathbf{1}^\top w = 1, \quad w \geq 0 \\ & w^\top \bar{r} \geq \bar{r}_{\min} \end{aligned}

Here $T$ is the number of historical scenarios $r^{(1)}, \ldots, r^{(T)}$ and $\bar{r}_{\min}$ is the target expected return. Three groups of variables: $w$ for asset weights, $\zeta$ for the VaR estimate, and one $u_t$ per scenario capturing the loss beyond VaR. The pair of constraints $u_t \geq 0$ and $u_t \geq \text{loss} - \zeta$ together implement $\max(\cdot, 0)$ , which is the standard trick for getting a piecewise-linear function into an LP.

Objective and constraints are linear in every variable, so this is a genuine linear program. The matrix is roughly $(T + N + 1) \times (N + T + 1)$ , comfortable for up to around 10,000 scenarios.

Three consequences are worth separating. It is convex, in fact an LP, so off-the-shelf solvers apply, you get a global optimum, and you never worry about local minima the way you do with Sharpe-ratio optimization, which is a fractional program and non-convex. It is distribution-free, optimizing over a discrete set of historical scenarios with no parametric assumption, so if the scenarios have fat tails it respects them and if the scenarios come from a stress period it captures the stress. And it is legible: the optimal $\zeta$ is your VaR, the optimal objective is your CVaR, the weights are your allocation, with no hidden regularization and no transformation.

Concretely, a working implementation with cvxpy:

import cvxpy as cp
import numpy as np

# Historical return scenarios: shape (T, N)
returns_scenarios = returns_hist.values  # T scenarios, N assets

T, N = returns_scenarios.shape
alpha = 0.05  # Worst 5%
target_return = 0.008  # 0.8% monthly

# Variables
w = cp.Variable(N)
zeta = cp.Variable()  # VaR estimate
u = cp.Variable(T)    # Excess losses

# Portfolio losses (positive = loss)
losses = -returns_scenarios @ w  # shape (T,)

# Objective: minimize CVaR
objective = cp.Minimize(zeta + 1.0 / (alpha * T) * cp.sum(u))

# Constraints
constraints = [
    u >= losses - zeta,    # u_t = max(loss_t - VaR, 0)
    u >= 0,
    cp.sum(w) == 1,        # Fully invested
    w >= 0,                # Long-only
    returns_scenarios.mean(axis=0) @ w >= target_return
]

problem = cp.Problem(objective, constraints)
problem.solve()

print("CVaR-optimal weights:", w.value.round(4))
print("Estimated VaR:", zeta.value)
print("Estimated CVaR:", problem.value)

Milliseconds to solve at typical sizes, and the output is a long-only portfolio tilted toward assets that hold up in the worst 5% of historical scenarios.

Put that portfolio next to the variance-optimal one built from the same data and the differences are systematic. The CVaR-optimal portfolio carries less left-tail risk, which is the direction it was optimized in, and it concentrates more in defensive assets, which is the same fact restated. Its mean return is usually lower, because variance penalizes right-tail blowups while CVaR neither penalizes nor chases them. Which one has the higher median depends on the scenario set; do not file that one as a rule. All of these are directional, not quantitative. Change the scenarios and the magnitudes move, and occasionally so does the sign.

What You Are Really Optimizing Is a Sample

There is no “true distribution” anywhere in that LP. The $\frac{1}{\alpha T}\sum u_t$ in the objective is the empirical CVaR, the average of the worst α% of historical scenarios. It is a sample average approximation (SAA) of the true CVaR, which would require knowing the underlying return distribution, which we do not.

The SAA converges to the true CVaR as $T \to \infty$ , there is a gap at finite $T$ , and the size of that gap is estimable:

\text{Var}(\widehat{\text{CVaR}}) \approx \frac{\sigma_{\text{tail}}^2}{\alpha T}

with $\sigma_{\text{tail}}^2$ the variance of the tail outcomes. The $\alpha$ in the denominator is the point: your effective sample size is not $T$ , it is $\alpha T$ . Halving the variance takes 4x the scenarios.

Practically this means use enough scenarios. A few hundred supports a stable CVaR estimate, a few thousand is better, and five years of daily returns gives roughly 1,260, which is usually enough. For more rigor you can add explicit regularization on the weights, an L2 penalty for instance, to absorb SAA noise. But be clear that this only squeezes the variance of the noise. It does nothing about bias in the scenarios themselves, which is the next problem.

The Scenarios Are the Real Input

The answer the optimizer gives you is determined entirely by the scenarios you feed it. In production that sentence carries more weight than all the math above it: change the sample window and the optimal weights change beyond recognition. Use 2008-2009 data and you get a defensive portfolio. Use 2017 data and you get a momentum tilt. The choice of scenario window matters as much as the choice of risk measure, and it is usually the one that never makes it into the methodology document.

So the scenarios should reflect the distribution of future returns rather than merely recite history. There are five common ways to generate them, each with its own flaw. Historical scenarios take a window of returns directly, giving a realistic dependence structure at the cost of limited scenarios and an implicit assumption that the future resembles the past. Filtered historical scenarios run historical returns through a GARCH or similar model to add volatility clustering, capturing regime-conditional dynamics at the cost of model dependence. Monte Carlo scenarios simulate from a fitted model (multivariate normal, t, skewed-t), giving unlimited scenarios at the cost of misspecification risk. Bootstrap scenarios resample historical returns with replacement, non-parametric but stuck with the same limits as the historical approach. Stress scenarios are hand-built for known risks (financial crisis, recession, inflation spike), making risk coverage explicit at the cost of subjectivity.

Most institutional systems mix them: filtered historical as the base, plus stress scenarios for known risks. The result is a scenario set that captures the statistical regularity without pretending the tail does not exist.

Constraints Are Nearly Free

CVaR optimization composes with other constraints, largely because most real constraints are already linear. Transaction costs $c^\top |w - w_{\text{current}}|$ are linear in weight changes and can go into either the constraints or the objective. Sector bounds $\sum_{j \in \text{sector}} w_j \leq b_{\text{sector}}$ are linear. A leverage limit $\sum_j |w_j| \leq L$ is linear, with absolute values that reduce to a plain sum when weights are non-negative. A turnover limit $\sum_j |w_j - w_{j,\text{current}}| \leq T_{\max}$ is the same story.

The one that breaks out is cardinality, $\sum_j \mathbb{1}[w_j > 0] \leq K$ , which is not convex and needs mixed-integer programming or a relaxation. The MILP version is computationally expensive, so most production systems drop it and use a soft version instead, penalizing non-zero weights in the objective.

Multi-period rebalancing connects too. Put the trading cost in the objective:

\min_{w, \zeta, u, v} \quad \zeta + \frac{1}{\alpha T} \sum_t u_t + c \sum_t v_t

subject to

\begin{aligned} & u_t \geq -w^\top r^{(t)} - \zeta, \quad \forall t \\ & u_t \geq 0, \quad \forall t \\ & v_t = \sum_i |w_i^{(t)} - w_i^{(t-1)}| \\ & \text{budget and other constraints} \end{aligned}

Still an LP, or a MILP once you allow discrete rebalancing decisions, with $c \sum_t v_t$ as the linear transaction cost penalty. Some institutional asset managers run exactly this, and its practical effect is to suppress over-rebalancing: even when the CVaR-optimal rebalance calls for large trades, the cost term asks first whether they are worth it.

The genuinely multi-period problem, where returns across periods are correlated and wealth compounds, has two routes. One simulates paths and optimizes over them: generate $K$ multi-period return paths, compute terminal wealth for each under the given weights, take the CVaR of terminal wealth, optimize. Expensive but flexible, since the CVaR can be taken of any quantity (terminal wealth, drawdown, whatever). The other formulates a stochastic program with recourse and solves it by decomposition (Benders, progressive hedging), which is cleaner in theory but needs specialized solvers. Most institutional systems take the first road for simplicity.

α Is Not a Field You Fill In Casually

$\alpha$ is the tail probability. Smaller means more risk-averse, staring at a more extreme tail; larger drifts toward mean-variance. The sensitivity here is much sharper than anything in variance optimization: at $\alpha = 0.01$ the optimizer is extremely risk-averse, and at $\alpha = 0.25$ , the worst quarter, its behavior is already close to mean-variance. No value is theoretically privileged, but four common settings each have a provenance. $\alpha = 0.01$ is very risk-averse; $\alpha = 0.025$ is the Basel III standard for market risk capital; $\alpha = 0.05$ is the common institutional default; $\alpha = 0.10$ is distinctly less conservative. Most institutional applications use 0.05 and risk capital calculations use 0.025 or 0.01. Institutions often run several values and blend, because committing to one point is itself an untested assumption.

The reason 0.05 became the default is that “the worst 5% of outcomes” on daily data is roughly 1-2 days per month, or a quarter every five years, which lines up with most institutional observation horizons. You have to accumulate enough tail events in reasonable time before verification is even on the table.

Keep one eye open when reading papers: $\alpha$ here is the lower-tail probability, and some authors define $\alpha$ as the confidence level instead, in which case every number flips. It is the same trap as the $1-\alpha$ in the VaR definition above.

Backtesting, Which Is Where This Is Thinnest

Mean-variance has a clean out-of-sample metric: realized Sharpe. CVaR has no clean equivalent, because realized CVaR depends on the worst α% of future outcomes, and that is a tiny sample. With daily returns and $\alpha = 0.05$ you get about 20 tail observations a year, and twenty points is not much to test a tail average with.

You still run the standard procedure:

Run the CVaR optimization in-sample.
Hold the portfolio out-of-sample.
Compute realized losses.
Compute realized VaR at level $\alpha$ and check the breach rate (it should be about $\alpha$ ).
Compute realized CVaR, the average loss in breach periods, and check it against the in-sample CVaR.

A breach rate far above $\alpha$ means your VaR model is bad. Realized CVaR far above in-sample CVaR means the model is miscalibrated. Christoffersen (1998) is the standard test for VaR; Berkowitz (2001), which tests the full distribution, is more rigorous.

The honest description of practice: most shops do not backtest CVaR rigorously. They backtest VaR, which is far easier, and assume that if VaR is fine CVaR is fine. That assumption breaks precisely when the shape of the tail changes, which is the reason you were using CVaR in the first place. Knowing you are making the trade-off is better than pretending you are not.

The same LP structure gets reused with a different definition of loss. CVaR measures loss over a fixed window; Conditional Drawdown (CDaR) measures drawdown from a peak. For a wealth process $W_t$ with running maximum $M_t = \max_{s \leq t} W_s$ and drawdown $D_t = M_t - W_t$ ,

\text{CDaR}_\alpha = \mathbb{E}[D_t \mid D_t \geq \text{DaR}_\alpha]

where DaR is drawdown-at-risk, the corresponding quantile of the drawdown distribution. For applications that genuinely care about the path, retirement portfolios among them, CDaR is the more relevant object, and its optimization is again a linear program with auxiliary variables. The Rockafellar-Uryasev trick works on any conditional tail expectation of this shape.

It Is Already the Regulatory Language

CVaR is now the dominant risk measure at central banks, sovereign wealth funds, and most large institutional asset managers, and Basel III uses Expected Shortfall, a CVaR variant, for market risk capital. Past that threshold, using it stops being a matter of taste: if you are building institutional risk systems, you need it.

What is worth reminding yourself of is the division of labor. The LP is the least error-prone link in the chain, being convex, globally optimal, millisecond-fast, and backed by mature libraries. The errors live upstream: the scenario window you picked, the $\alpha$ you picked, the tail you cannot rigorously backtest. The optimizer will not carry those choices for you. It will only amplify them into weights.

References

Rockafellar & Uryasev (2000). “Optimization of Conditional Value-at-Risk.” Journal of Risk 2(3): 21-41. The LP formulation. The seminal paper.
Rockafellar & Uryasev (2002). “Conditional Value-at-Risk for General Loss Distributions.” Journal of Banking & Finance 26(7): 1443-1471. Extension to non-normal distributions.
Artzner, Delbaen, Eber & Heath (1999). “Coherent Measures of Risk.” Mathematical Finance 9(3): 203-228. The coherence axioms.
Acerbi & Tasche (2002). “Expected Shortfall: A Natural Coherent Alternative to Value at Risk.” Economic Notes 31(2): 379-388.
Krokhmal, Palmquist & Uryasev (2002). “Portfolio Optimization with Conditional Value-At-Risk Objective and Constraints.” Journal of Risk 4: 43-68. The full portfolio formulation.
Sarykalin, Serraino & Uryasev (2008). “Value-at-Risk vs. Conditional Value-at-Risk: A Comparison.” In Risk Management: Value at Risk and Beyond. Cambridge University Press.
Kritzman, Page & Turkington (2012). “Regime Shifts: Implications for Dynamic Strategies.” Financial Analysts Journal 68(3). Volatility versus tail risk across regimes.
Basel Committee on Banking Supervision (2016). “Minimum Capital Requirements for Market Risk.” Bank for International Settlements. The regulatory adoption.

Next article: resampled efficiency. Michaud’s bootstrap approach, formalized for institutional use.