Multi-Period Portfolio Optimization, the Li-Ng Route

Every earlier article in this module assumed a single period: set weights today, hold for one period, observe returns, rebalance. Whether “one period” means a day, a month or a year does not change the structure.

Portfolios are not managed that way. Today’s allocation determines what you can do tomorrow. Selling A to buy B when you expect to rebalance next month means something different from making the same trade when you intend to hold for five years. Multi-period optimization writes that dependence into the objective: the mean-variance trade-off is evaluated over the entire horizon $T$ rather than over the next period alone.

The trouble is that once “multi-period” goes into the objective, dynamic programming stops working — not becomes expensive, stops working. Bellman’s principle fails outright. That is the real contribution of Li and Ng (2000): they did not solve the problem, they replaced it with one that is solvable and then proved the solution you wanted lives inside it. This article works through that substitution, follows it into continuous time, and closes with why production systems still run myopic approximations.

Why Dynamic Programming Does Not Apply

Take an investor with initial wealth $W_0$ over a horizon of $T$ periods. At each period $t$ they observe the state and choose weights $w_t$; returns $r_t$ realize and wealth evolves as

subject to that wealth dynamic plus any position constraints (no shorting, a budget constraint, and so on). The objective is a mean-variance trade-off on terminal wealth:

The variance is where it breaks. $\mathrm{Var}(W_T) = \mathbb{E}[W_T^2] - (\mathbb{E}[W_T])^2$, and that squared expectation is not the expectation of anything. Dynamic programming requires the objective to decompose into nested conditional expectations, and $(\mathbb{E}[W_T])^2$ refuses: the optimal continuation from any interior period depends on the unconditional mean you committed to at time 0. Put differently, the policy computed today is no longer optimal when recomputed tomorrow from the new information set. That is time inconsistency, not a computational difficulty.

Li and Ng’s Substitution

Their move is to introduce an auxiliary problem:

It differs from the original by exactly one term — expand it and the auxiliary objective equals the mean-variance objective minus $\frac{\gamma}{2}(\mathbb{E}[W_T])^2$. But the term has changed position: the whole objective is now the expectation of one function of $W_T$, it decomposes period by period, Bellman’s principle is restored, and dynamic programming proceeds normally.

What remains is joining the two problems back up. Li and Ng prove the embedding relation: every efficient solution of the original mean-variance problem solves the auxiliary problem for some value of its parameter. Sweep the auxiliary parameter and the efficient frontier appears among the results; to land on the particular point you want, solve one scalar equation linking the parameter to the realized $\mathbb{E}[W_T]$. That is the entire trick — not solving a hard problem, but embedding it in a family of solvable ones and picking the answer back out. Everything below works on the auxiliary problem.

A normalization makes the algebra cleaner. Define $y_t$, which evolves proportionally with wealth:

With $y_0 = 1$ this gives $y_t = W_t / W_0$, the cumulative return multiplier. Expected terminal wealth and its second moment become

and

so the auxiliary objective is

$y_T$ is badly nonlinear along the return path — it is a product of random factors — but that does not matter here. The distribution of $y_T$ depends only on the weight sequence $w_0, \ldots, w_{T-1}$, and the objective uses only its first two moments. So $y_t$ serves directly as the state variable.

The Backward Recursion

Define the value function:

with the corresponding Bellman equation

and terminal condition $V_T(y) = y - \frac{\gamma W_0}{2} y^2$.

Whether the recursion is computable depends on whether $V_t$ keeps a fixed functional form. Here it does: the terminal condition is quadratic and the wealth dynamic is linear in $y$, so the quadratic form is closed under the recursion,

with coefficients $(a_t, b_t, c_t)$ depending on $t$ but not on $y$. Substituting back into the Bellman equation and matching powers of $y$ produces a backward recursion for those coefficients, started from the terminal condition:

Each backward step does two things: solve a single-period MVO with the period’s conditional mean $\mu_t$ and covariance $\Sigma_t$ to get $w_t^\star$, then use it to update the coefficients. Under the quadratic value function, the optimal weight takes the form

where $c_{t+1} < 0$, so the leading scalar is positive. Notice that the state $y_t$ enters only as the scalar $1/y_t$ outside the bracket — the vector inside does not depend on current wealth at all. That is the main convenience the quadratic value function buys: every backward step reduces to one vector equation instead of a search over the state space. The closed-form coefficient updates are in Li and Ng’s original paper, and copying them is faster than rederiving them.

What the Optimal Policy Looks Like

Written more plainly, the structure is

$\Sigma_t^{-1}\mu_t$ is the familiar single-period MVO direction, the leading scalar varies with time, and the intertemporal term is what belongs to the multi-period problem alone: it captures hedging demand against future periods. The reason it is nonzero is mundane — the money you commit today changes tomorrow’s opportunity set, and the value function can see that. At $T = 1$ the term vanishes and the expression collapses back to standard MVO. The larger $T$ is, the more it matters.

That $1/y_t$ factor deserves its own remark. After wealth rises, the same bracketed vector is divided by a larger $y_t$, so the risky position shrinks. Multi-period mean-variance therefore behaves less like a proportional allocation and more like something walking toward a finish line: take more risk while the target is far away, pull back as it comes into reach. This is a different animal from fixed-proportion single-period MVO, and it is the same pattern that reappears in the continuous-time solution below.

State Variables and the Curse of Dimensionality

In the basic Li-Ng formulation, wealth is the only state. In practice there are more: current wealth $y_t$, current holdings (which determine where transaction costs start from), a time-varying expected return $\mu_t$, a time-varying covariance $\Sigma_t$. Each additional state variable adds a dimension to the dynamic program; the value function stays quadratic in the wealth direction but need not stay tractable in the others. Extensions by Björk, Murgoci and Zhou and others handle multi-factor models, state-dependent risk aversion, and constraints.

Once the state representation is fixed, the multi-period problem is a standard Markov decision process solved the usual way: discretize the continuous state space, compute the value function backward from $t = T$ to $t = 0$, extract the optimal policy $w_t^\star(x)$ for each state and time, then at runtime observe the state, look up the policy and execute. With discrete states the recursion is exact; with continuous states the value function has to be approximated by polynomial bases, neural networks or kernel methods.

Dimensionality is a hard wall here. Beyond three or four continuous state variables, dynamic programming is infeasible without strong structure. Three routes around it: assume the value function belongs to some parametric family (quadratic, polynomial) and estimate the parameters; simulate many paths and estimate the value function by regression; or treat it as an MDP outright and learn the policy with reinforcement learning. The latter two scale better in high dimensions at the cost of far more data. For the typical portfolio problem — few states, moderate horizon — direct dynamic programming is still workable. High-frequency settings or very long horizons are where simulation-based methods become necessary.

The Continuous-Time Limit

As the rebalancing frequency goes to infinity, the discrete-time Li-Ng problem converges to a continuous-time stochastic control problem. The objective is still mean-variance on terminal wealth:

and the same embedding converts it into a solvable auxiliary objective. Wealth evolves as

where $Z_t$ is Brownian motion and $r_f$ is the risk-free rate. The solution runs through the Hamilton-Jacobi-Bellman equation:

The first-order condition in $w$ gives

Here $-W V_{WW} / V_W$ is the effective relative risk aversion, varying with time and wealth. That is precisely the structural difference from the single-period problem: single-period MVO takes risk aversion as an exogenous constant, while in the multi-period problem it is a product of the value function’s curvature.

Zhou and Li (2000) give the explicit solution in a linear-quadratic framework. It reads most clearly as the dollar amount $\pi_t^\star$ held in risky assets rather than as weights:

with $\zeta$ a constant fixed by the target expected terminal wealth. Risk exposure is proportional to how far current wealth still is from the target: far from it, take more risk; approaching it, take less; exactly at it, take none. This is the full version of the $1/y_t$ factor from the discrete case, and it exposes the least intuitive property of mean-variance over multiple periods — once wealth exceeds the target, the bracket turns negative and the optimal policy calls for a reversed position in risky assets. A variance penalty does not distinguish upside from downside; overshooting is deviation too. Understand that consequence before adopting the objective.

Back to Discrete Time

Real systems run in discrete time, so the discrete Li-Ng problem can be read as an approximation to the continuous one. Daily rebalancing over a one-year horizon gives $T = 252$ and a fairly close approximation; monthly rebalancing over five years gives $T = 60$ and a much coarser one.

The rebalancing frequency is itself something to optimize: rebalance more often and you track the continuous-time optimum more closely while paying more in transaction costs. The optimal frequency balances the two, and it depends on your cost structure, so there is no universal answer.

Adding Transaction Costs

The previous article covered transaction-cost-aware optimization. The multi-period framework accommodates it naturally, since each period’s cost depends on the old and new weights and the multi-period problem already looks at consecutive periods together. In discrete time:

still subject to the wealth dynamic. The price is that current holdings must join the state space, the value function no longer depends on wealth alone, and an exact solution requires dynamic programming over the full state space, which is expensive.

Gârleanu and Pedersen (2013) give what is essentially the continuous-time version of this problem: heavier mathematics, but a clean conclusion — the optimal policy converges partially toward a target portfolio, at a rate set jointly by costs and the decay rate of the signal. For practitioners the link between the discrete and continuous routes is mostly theoretical; implemented carefully, both give similar results, and the choice comes down to the tooling your institution already has. Most institutions in fact run the myopic approximation: solve a single-period MVO with a transaction-cost penalty each period, discarding the intertemporal structure in exchange for simplicity. Only when the stakes are large enough — pension funds, sovereign wealth funds — does the full multi-period solution justify its compute bill.

Regime Switching and Liabilities

Two common extensions, both of the “widen the state space” variety.

Returns often show regime-switching behavior: bull and bear, high volatility and low. Standard Li-Ng assumes a single regime; the extension makes the moments regime-dependent,

where $z_t$ is a Markov chain on the regime space. The investor observes the current regime and chooses $w_t$ accordingly, and the recursion becomes

with $z'$ generated by the chain’s transition probabilities. Solving it requires the transition probabilities and the regime-conditional moments, usually estimated from history with a hidden Markov model or similar. The upside is a portfolio that adapts to the current regime; the cost is a whole additional set of parameters to estimate, and regime identification itself lags.

The other extension is liabilities. A pension fund has a stream of future obligations and a pool of assets, and the goal is not to maximize wealth but to cover the liabilities at minimum cost. Generalizing Li-Ng, the state space holds both asset wealth $W_t$ and the present value of liabilities $L_t$, and the objective becomes minimizing expected shortfall of wealth against liabilities. This is liability-driven investment, and the optimization balances a low probability of shortfall against a high expected return. In production it is implemented as a multi-stage stochastic program, with the number of stages set by the liability structure — 20 to 40 annual stages for a typical pension fund.

The Price of Quadratic Utility

Li-Ng uses quadratic utility, which is where mean-variance comes from and which carries a well-known defect: a downward-opening parabola has negative marginal utility past its peak, meaning more wealth makes you worse off. Quadratic utility is therefore meaningful only below that satiation point, and you should confirm the wealth range you care about sits inside it. In most portfolio applications this does not bind, but it is a genuine theoretical flaw and not worth pretending away.

Several alternatives exist, each producing a different optimal policy. Log utility, $U(W) = \log W$, has strictly positive marginal utility and is the standard choice in the continuous-time literature. Power utility, $U(W) = W^{1-\gamma}/(1-\gamma)$, gives CRRA preferences. Exponential utility, $U(W) = -\exp(-\gamma W)$, gives CARA preferences and is common in institutional risk management. Mean-variance is the most widely used in portfolio optimization; it is neither the only option nor the best-motivated one.

What Production Does, and Why Myopic Often Wins

Full multi-period solutions are rare in production. Three approximations are common. The simplest is myopic optimization: solve a single-period MVO each period with current estimates and ignore the multi-period structure entirely. For long-horizon, low-turnover investors the loss is smaller than you would guess. The second solves the multi-period problem once at the start, assuming constant $\mu$ and $\Sigma$, and holds the resulting policy for the whole horizon — buy-and-hold with an unusually thoughtful initial allocation. The third is rolling multi-period: re-solve the multi-period problem at each rebalance over a shorter horizon, say twelve months. It sits between the other two and is what most institutions that adopt multi-period actually run.

In code, a minimal multi-period optimizer looks roughly like this:

import numpy as np

def multi_period_mvo(mu, Sigma, T, gamma, W0):
    """
    Li-Ng multi-period mean-variance.
    mu: (T, N) expected returns for each period
    Sigma: (T, N, N) covariances for each period
    T: horizon length
    gamma: risk aversion
    W0: initial wealth
    """
    N = len(mu[0])
    
    # Backward recursion for value function coefficients
    # V_t(y) = a_t + b_t * y + c_t * y^2
    c = np.zeros(T + 1)
    b = np.zeros(T + 1)
    a = np.zeros(T + 1)
    
    c[T] = -gamma * W0 / 2
    b[T] = 1
    a[T] = 0
    
    # Precompute inverse covariances
    Sigma_inv = [np.linalg.inv(Sigma[t]) for t in range(T)]
    
    weights = []
    for t in range(T - 1, -1, -1):
        # Optimal weight at time t
        w_t = -1 / (gamma * W0) * Sigma_inv[t] @ mu[t]  # + intertemporal term
        # (Intertemporal term omitted for clarity)
        weights.append(w_t)
    
    return list(reversed(weights))

The “intertemporal term omitted” comment is not a convenient dodge; it is the entire distance between this code and actual Li-Ng. The full version has to run the backward recursion on $(a_t, b_t, c_t)$ before hedging demand appears at all. What the snippet above computes is a period-by-period myopic solution.

So where does multi-period genuinely beat repeated single-period? In four places. It sees how today’s trade changes tomorrow’s opportunity set, which is the hedging term. It handles $\mu_t$ and $\Sigma_t$ that vary over time, whereas single-period only ever uses the current estimate. It can model risk aversion that changes with wealth, where single-period holds it fixed. And both can handle transaction costs, but multi-period handles them more naturally because it can see the future trajectory. The differences show up most at short horizons, high turnover, or strong intertemporal dependence — pension liability-driven investment being the canonical case.

Theoretically better is not the same as better in practice, and the myopic approximation wins more often than its critics admit. First on the list is accumulated estimation error: every period’s $\mu_t$ and $\Sigma_t$ is estimated, and the multi-period solution compounds those errors along the horizon, making it more sensitive to them than the single-period one. Second is model risk — the multi-period formulation assumes you know the data-generating process, or at least the conditional distribution of returns; a wrong model gives a wrong policy, and you have baked in a layer of intertemporal dependence that may not exist. Third is compute: myopic is fast, multi-period is not. Fourth is behavioral realism: real investors do not optimize against a 60-month horizon, and the trajectories the multi-period solution produces often do not match what an institution can actually stick to.

The honest conclusion is that multi-period MVO is the theoretically correct framework whose implementation benefits frequently fall short of its costs. High stakes, long horizons and explicit liability structures (pension funds, sovereign wealth funds) justify the expense. Short-horizon, high-turnover strategies — high-frequency, market-neutral — are fine with myopic.

The Frontier

Recent work extends the problem in four directions. Treating portfolio management as an MDP and learning the policy and value function with deep reinforcement learning (Jiang, Xu & Liang, 2017; Deng et al., 2017). Replacing mean-variance with multi-period CVaR, which remains tractable by dynamic programming once auxiliary variables are introduced. Combining robust optimization with the multi-period setting, optimizing the worst case over an uncertainty set and over future periods. And integrating transaction costs fully into the multi-period framework, where the continuous-time formulation is cleanest.

All four have clear theoretical motivation and all four are hard to land. The frontier of multi-period portfolio optimization is really the frontier of stochastic control under realistic constraints.

Closing

This is the last article in the portfolio construction module. Twelve articles ran from Markowitz to multi-period optimization, by way of estimation error, shrinkage, robust optimization, risk parity, hierarchical risk parity, CVaR, Black-Litterman, transaction costs and factor construction.

If one thing survives from this article: the difficulty in multi-period mean-variance is not computational. The mean-variance objective is incompatible with dynamic programming, and Li-Ng’s response — substitute a separable objective, solve it, map back — is a general move in stochastic control. That move is the thing worth remembering, not the coefficient recursion.

Literature

• Li & Ng (2000). “Optimal Dynamic Portfolio Selection: Multi-Period Mean-Variance Formulation.” Mathematical Finance 10(3): 387-406. The seminal paper.
• Markowitz (1959). Portfolio Selection: Efficient Diversification of Investments. Wiley. Mean-variance foundations.
• Björk, Murgoci & Zhou (2014). “Mean-Variance Portfolio Optimization with State-Dependent Risk Aversion.” Mathematical Finance 24(1): 1-24. State-dependent extension.
• Zhou & Li (2000). “Continuous-Time Mean-Variance Portfolio Selection: A Stochastic LQ Framework.” Applied Mathematics and Optimization 42(1): 19-33. The continuous-time version.
• Cui, Li, Lu & Li (2012). “Multi-Period Mean-Variance Portfolio Optimization with Markovian Regime Switching.” Journal of Mathematical Analysis and Applications 387(1): 101-109.
• Gârleanu & Pedersen (2013). “Dynamic Trading with Predictable Returns and Transaction Costs.” Journal of Finance 68(6): 2309-2340. Continuous time with costs.