CPCV: Combinatorial Purged Cross-Validation, Properly Derived

Standard k-fold cross-validation is wrong for financial machine learning, and there is exactly one reason: observations are glued together in time. The label at time $t$ reaches forward past $t$, the features at time $t$ reach backward before $t$, so wherever you cut, the two sides of the cut are still connected. Meanwhile the entire theory of k-fold CV rests on the folds being independent.

Everything derived below follows from that sentence: purging, embargoing, the path counting formula $\varphi[N,k]$, and the disagreement between CPCV and walk-forward analysis. There is also a price at the end that introductions to CPCV often skip: to buy more paths, CPCV lets the model train on the future to predict the past.

How the Leak Happens

Standard k-fold CV cuts the data into $k$ contiguous blocks, trains on $k-1$ of them, tests on the one held out, and rotates. In a backtest the pipeline is:

1. Compute features $X_t$ from data before $t$.
2. Compute the label $y_t$ from data between $t$ and $t + h$ (a forward return, say).
3. Train a model to predict $y_t$ from $X_t$.
4. Test on out-of-fold data.

Step 2 is where it goes wrong: $y_t$ uses information after $t$, so the training set can contain observations whose labels reach into the test set.

Concretely. Suppose the features include a 20-day moving average and the label is a 5-day forward return. The training set is days 1 through 252 and the test set is days 253 through 378. The label for training observation day 250 uses returns from days 250 through 255. The label for day 252 uses days 252 through 257. Days 253 through 257 are in the test set.

So the model has already seen the first few days of the test period during training, through its labels. It then performs better at test time than it deserves to, and that improvement is pure memorization. The test metric is biased upward. That is the leak.

Purging: The Labels That Reach Forward

The fix is direct: delete from the training set any observation whose label overlaps the test set. That operation is purging.

Let the test fold run from index $t_{start}$ to $t_{end}$, with label horizon $h$. A training observation $t$ has a label ending at $t + h$, and if that reaches the start of the test set, it goes:

Both conditions are needed. The first restricts attention to training observations before the test fold; the second is the actual overlap test. Writing only the second would wrongly delete the entire stretch of training data after the test set as well.

With the numbers above, $h = 5$ and $t_{start} = 253$, purging removes training observations with $t \ge 248$, which is days 248, 249, 250, 251, and 252. Five observations.

The purge length equals the label horizon $h$, and that is a floor. Longer labels cost more: a 21-day forward return means cutting 21 observations before every test fold.

Embargo: The Features That Reach Backward

Purging plugs one direction only. Labels reach forward, features reach backward, and the second direction leaks just as readily.

Consider the training observations immediately after the test fold ends. A 20-day moving average computed at $t_{end} + 3$ spans $t_{end} - 17$ through $t_{end} + 3$, and most of that window sits inside the test period. Putting that observation into the training set feeds test-period prices to the model in the shape of a feature.

The embargo removes a short stretch of training observations after the test fold:

where $e$ is the embargo length. The embargo applies only after the test fold, because the side before it is already handled by purging. The two operations point in opposite directions and each covers one end; together they close the known channels between training and test.

How to set $e$? For a strategy with a short label horizon and short feature windows, $e = h$ suffices. The more general rule of thumb is

where $w$ is the longest rolling window in your features, which covers both the label dependence and the feature dependence. Some implementations simply take 1% to 5% of the series, which is easy but ignores the actual features, a crude conservatism. Setting $e = 0$ means no embargo at all.

There is a less standard use worth knowing about: set $e$ to the typical length of a market regime, forcing training and test data to come from the same regime. If the test period is in a high-volatility regime while the training set mixes in low-volatility data, the relationship the model learns may simply not apply. The cost is fewer usable observations per split; the benefit is a cross-validation that matches the question you actually care about.

From One Split to Combinatorial Splits

Purging and embargoing solve the leak, but not a second problem: standard k-fold gives you only $k$ test paths, which is too small a sample to judge how stable “out-of-sample Sharpe of 1.2” really is.

CPCV’s idea is to make the splitting combinatorial. In López de Prado’s notation, cut the series into $N$ contiguous groups and, in each split, hold out $k$ of them for testing while the remaining $N-k$ train. The number of splits is then

At this point splits and paths have to be kept apart, because they are different numbers in the original literature and conflating them wrecks everything downstream.

A single split only covers part of the timeline, namely its $k$ test groups. To assemble a backtest path that runs the full timeline, you need one test record for every group. Each group is tested in $\binom{N-1}{k-1}$ of the splits, and that count is exactly the number of complete paths you can build:

The two expressions are equal, and a quick check confirms it. With $N = 6$ and $k = 2$, there are $\binom{6}{2} = 15$ splits and $\frac{2}{6} \times 15 = 5$ paths, which is also $\binom{5}{1} = 5$. This is the canonical six-groups-two-tested example from AFML: 15 training runs buying 5 complete out-of-sample equity curves.

Compare walk-forward on the same data: it gives you 1. That jump from 1 to 5 is the entire selling point of CPCV.

Some common configurations:

At $k = 2$ the path count is just $N - 1$, growing linearly, while the split count (the number of times you have to train a model) grows quadratically. That divergence sets the practical trade-off: one more path costs another batch of model fits.

The Algorithm

Step 1: Partition the data into $N$ contiguous, non-overlapping groups.

Step 2: Enumerate every size-$k$ combination of groups as the test set, with the remaining $N-k$ groups as training.

Step 3: For each test group’s start $t_{start}$, purge training observations in $[t_{start} - h, t_{start})$.

Step 4: For each test group’s end $t_{end}$, embargo training observations in $(t_{end}, t_{end} + e]$.

Step 5: Train the model on what remains after purging and embargoing.

Step 6: Predict on the test groups and record performance.

Step 7: Repeat over all $\binom{N}{k}$ splits.

Step 8: Assemble the per-group test predictions into $\varphi[N,k]$ complete paths, giving a distribution of performance.

Step 8 is what separates CPCV from “running cross-validation a lot of times.” What you want is not $\binom{N}{k}$ isolated scores but $\varphi[N,k]$ complete equity curves, each with its own Sharpe and its own maximum drawdown.

Aggregating: Concatenate or Average

With multiple paths in hand there are two ways to summarize, and they answer different questions.

Concatenating strings together all test segments along a path and computes performance once over that full series. It answers “if this strategy ran across the whole timeline, what happened?”

Averaging computes a performance number per path first, then averages across paths. It answers “what does a typical path look like?”

Look at both. Note that the concatenated Sharpe and the average of the per-path Sharpes are generally not equal, and there is no simple conversion between them: the concatenated Sharpe comes from the overall mean and overall standard deviation of one long series, and a single high-volatility stretch moves both the numerator and the denominator. When the two numbers diverge sharply, the paths differ a lot from each other, which is itself useful information.

More informative than either number is the full distribution. The mean tells you expected performance if you deploy into a randomly chosen market condition. The variance tells you how much to trust that expectation. The worst few paths set your risk budget. Negative skew warns that occasional stretches will look ugly.

Together those constitute the deployment judgment. A strategy averaging Sharpe 1.0 with a cross-path standard deviation of 0.5 is harder to ship than one averaging 0.8 with a standard deviation of 0.2, because with the first you do not know which outcome you are going to get. CPCV hands you that distribution directly, and that is its most concrete advantage over a single-path walk-forward.

A Worked Example

Take $T = 2000$ observations (roughly 8 years of daily data), cut into $N = 6$ groups of about 333 observations each (about 1.3 years). Each split tests $k = 2$ groups.

The label horizon is $h = 5$ (5-day forward returns), the longest feature window is $w = 60$ (a 60-day moving average), so the embargo is $e = \max(5, 60) = 60$.

That gives $\binom{6}{2} = 15$ splits, $\varphi[6,2] = 5$ paths, and each group tested 5 times across the 15 splits.

Take one concrete split, with test groups covering days 333–665 and days 1332–1664:

• Purge: drop training days 328–332 and days 1327–1331. Five observations each.
• Embargo: drop training days 666–725 and days 1665–1724. Sixty observations each.

This split discards 130 observations in total. The test set holds about 666 observations, and the training set falls from a nominal 1332 to about 1202.

Across all 15 splits, the average is about 87 observations discarded per split, which is 4.3% of the whole series and roughly 6.5% of the nominal training data. The amount varies with whether the two test groups are adjacent: when they are, one embargo region falls inside the other test group, which was never training data anyway, so less is wasted.

That 6.5% is the price of correctness. What it buys is the absence of any known information channel between training and test.

An Implementation That Runs

import numpy as np
from itertools import combinations
from math import comb


def cpcv_splits(T, n_groups=6, n_test=2, label_horizon=5, embargo=60):
    """
    Generate CPCV train/test splits with purging and embargo.

    Yields (train_idx, test_idx) for each of C(n_groups, n_test) splits.
    The number of complete backtest paths these splits support is
    phi[N, k] = k * C(N, k) / N = C(N - 1, k - 1).

    Parameters
    ----------
    T : int
        Number of observations.
    n_groups : int
        N, the number of contiguous groups the series is cut into.
    n_test : int
        k, how many groups are held out for testing in each split.
    label_horizon : int
        h, how far forward a label reaches. Drives purging.
    embargo : int
        e, how many observations after a test group are dropped from training.
    """
    g = T // n_groups
    bounds = [(i * g, (i + 1) * g if i < n_groups - 1 else T) for i in range(n_groups)]

    for test_groups in combinations(range(n_groups), n_test):
        blocked = np.zeros(T, dtype=bool)
        test_idx = np.concatenate([np.arange(*bounds[j]) for j in test_groups])
        blocked[test_idx] = True
        for j in test_groups:
            t_start, t_end = bounds[j]
            blocked[max(0, t_start - label_horizon):t_start] = True   # purge
            blocked[t_end:min(T, t_end + embargo)] = True             # embargo
        yield np.flatnonzero(~blocked), np.sort(test_idx)


T, N, k = 2000, 6, 2
splits = list(cpcv_splits(T, n_groups=N, n_test=k, label_horizon=5, embargo=60))

print(f"splits: {len(splits)} (= C({N},{k}) = {comb(N, k)})")
print(f"backtest paths: {k * comb(N, k) // N} (= C({N-1},{k-1}) = {comb(N-1, k-1)})")
for i, (train_idx, test_idx) in enumerate(splits[:3], start=1):
    dropped = T - len(train_idx) - len(test_idx)
    print(f"  split {i}: train {len(train_idx)}, test {len(test_idx)}, dropped {dropped}")

This prints 15 splits and 5 paths, with the first three training sets sized 1274, 1209, and 1209. The first split discards noticeably less (60 observations against 125) because its two test groups happen to be adjacent.

The blocked boolean mask is the key to this implementation: purging and embargoing just mark the mask, and the complement is taken once at the end. That is far easier to write than repeated set differences on index lists, and much harder to get wrong at the boundaries.

CPCV Versus CSCV

The two acronyms differ by one letter and describe different things, and confusing them is a common error.

CSCV assumes i.i.d. observations, does no purging and no embargoing, uses symmetric equal-mass splits to compute out-of-sample ranks, and feeds PBO, which measures whether your strategy selection procedure is overfit. CPCV purges and embargoes, and feeds a distribution of test performance, which measures what out-of-sample performance to expect from a model and how stable it is.

So the criterion is what you want to ask. To find out whether “picking the best of a batch of candidates” is a trustworthy act, use CSCV and compute PBO. To find out how a model is likely to perform once deployed, use CPCV. They do not compete, and a serious pipeline runs both.

CPCV Versus Walk-Forward: A Real Trade-off

You will sometimes read that walk-forward analysis is a special case of CPCV. It is not, and the claim hides something important.

Walk-forward strictly preserves temporal order: always train on the past and test on the future. CPCV does not. Among the $\binom{N}{k}$ splits, many put training groups after the test groups. When group 5 is the test set and groups 1 through 4 plus group 6 are training, the model is using group 6 (the future) to predict group 5 (the past).

That is not an implementation flaw; it is CPCV’s premise. It assumes the data-generating process is stable enough over the sample that training on the future and training on the past teach the same relationships. What that assumption buys is paths, going from walk-forward’s one to $\varphi[N,k]$.

When the assumption holds, this is a good trade: the same data yields a distribution rather than a point estimate. When it does not hold, with genuine structural breaks, regulatory changes, or market microstructure reforms, CPCV will return more optimistic results than walk-forward, because it let the model see things it could not have seen at the real point in time.

The practical answer is to run both. CPCV gives you the distribution and the statistical power; walk-forward gives you an honest benchmark with no time travel in it. When they agree, you can relax. When walk-forward is clearly worse, that is usually a signal that your data-generating process is less stationary than you assumed.

Computational Cost

CPCV is expensive, and the expense sits in the split count, not the path count. At $N = 10$ and $k = 2$ you get 9 paths but train 45 models; at $N = 16$ and $k = 2$, 15 paths cost 120 fits. For gradient boosting or deep learning that multiplier quickly becomes the difference between an overnight run and a week-long one.

A few workable compromises: keep $N$ between 6 and 10 so the split count stays in the dozens; run CPCV exploration with fast models (linear, shallow trees) and reserve the slow model for the final configuration; and parallelize, since the splits are fully independent.

Hyperparameter selection is the same arithmetic. The principled approach runs CPCV for every configuration in the grid and picks the best average test performance, which is more honest than standard cross-validation (no leakage) at a cost of grid size times split count. With 5 to 20 configurations that is tolerable. With a deep model carrying millions of parameters it is out of the question, and the move there is to search with a fast proxy (a single validation set) and then run CPCV once on the final configuration to confirm.

What It Guarantees, and What It Does Not

CPCV’s core guarantee is that after purging and embargoing, given a known label horizon and known feature windows, no information channel connects the training and test sets. The clause in bold is the whole set of conditions.

The independence is approximate, and residual dependence comes from at least three sources: volatility clustering that persists past the embargo length; structural breaks in the data-generating process; and feature dependencies you did not think of and therefore did not count in $w$. That last one is the dangerous one, because it is silent. A feature that quietly uses a 250-day window, paired with a 60-day embargo, leaks exactly as before and nothing raises an error.

There is a cheap check: look at the autocorrelation of the test-period residuals. If they show significant autocorrelation (Ljung-Box or similar), the embargo is probably too short. That is far quicker than auditing every feature by hand.

More importantly, CPCV solves leakage and only leakage. It does nothing about execution costs, capacity constraints, or regime change. A model with excellent CPCV performance can still fail live because trading costs consume the entire margin, or because market impact runs away once the size scales up.

So a rigorous deployment judgment is layered: get the test performance distribution from a properly purged and embargoed CPCV, then haircut it in turn for capacity, execution cost, and regime, and decide from the haircut distribution. CPCV is only the first step, and not one of the later steps can be replaced by it.

Where It Comes From

CPCV has its roots in the time series cross-validation literature. Bergmeir and Benítez (2012) systematically examined how much bias standard cross-validation introduces when evaluating time series predictors, and it is a standard citation on that thread; Hyndman and Athanasopoulos’s Forecasting: Principles and Practice gives the textbook treatment of time series cross-validation. Purging and embargoing themselves are practitioner inventions aimed at financial machine learning, addressing two problems that barely surface in the general time series literature: overlapping labels and feature windows.

For research papers in finance, CPCV is becoming the default expectation. The method is clear, the results are reproducible, and declining to report it is usually a signal in itself. If you are writing such a paper, report the full distribution of test performance rather than only the mean: the distribution is the diagnostic, the mean is just the summary.

References

• López de Prado, M. (2018). Advances in Financial Machine Learning. Wiley. ISBN 978-1-119-48208-6. Chapter 7 covers purging and embargoing; Chapter 12 covers backtesting through cross-validation (CPCV). The primary reference.
• Bailey, D. H., Borwein, J., López de Prado, M., and Zhu, Q. J. (2017). “The Probability of Backtest Overfitting.” Journal of Computational Finance, 20(4), 39–70. The CSCV variant.
• Bergmeir, C. and Benítez, J. M. (2012). “On the use of cross-validation for time series predictor evaluation.” Information Sciences, 191, 192–213.
• Hyndman, R. J. and Athanasopoulos, G. Forecasting: Principles and Practice. OTexts. The standard treatment of time series cross-validation.
• Aronson, D. (2006). Evidence-Based Technical Analysis, Chapters 6, 9–12. Wiley. ISBN 978-0-470-00874-4. Statistical foundations for hypothesis testing in technical analysis.
• Harvey, C. R., Liu, Y., and Zhu, H. (2016). “…and the Cross-Section of Expected Returns.” Review of Financial Studies, 29(1), 5–68. Multiple testing in factor research.