Walkthrough 01: Purged K-Fold Cross-Validation — From Novice to Proficient

Goal: By the end, you’ll understand why standard cross-validation lies to you about trading strategies, how purged K-fold fixes it, and how to implement it correctly.

Prerequisites: Basic familiarity with cross-validation (train/test splits) and financial time series.

Step 1: What Is This, Really?

Let’s say you have 5 years of daily price data for a mean-reversion strategy. You want to know if it actually works.

You do what any ML practitioner would do: split into training and test sets, train on the first 4 years, test on the last year. You get a Sharpe of 1.8. You feel good.

Then you go live. Three months in, the strategy is bleeding. Your Sharpe is -0.3.

What happened?

The training set has information about the test set. Not in an obvious way — there’s no “future data” literally in the training set. But the data is autocorrelated. Prices on day 800 are correlated with prices on day 801. When you train on days 1-800 and test on days 801-1000, your model has learned patterns that exist in both periods. It looks better than it is.

Standard cross-validation assumes your samples are independent. Financial data violates this assumption constantly. Purged K-fold is a modification of cross-validation that accounts for serial correlation.

Step 2: Why Does This Matter?

Here’s what happens without purged cross-validation:

You will overestimate your strategy’s performance. This isn’t a small bias. In a market with momentum (most markets), the bias is substantial. You might think your Sharpe is 1.8 when it’s really 0.6.

You will select the wrong strategy. If you’re comparing 10 strategy variants, the one that looks best in backtest might just be the one that benefited most from information leakage. You’re not selecting the best strategy — you’re selecting the luckiest.

You will lose money. This is the consequence. Going live with an overfitted strategy means you’re trading a model that doesn’t work on unseen data. Your drawdowns will be worse than expected, and your Sharpe will be lower than backtested.

The trap is that the backtest looks perfectly clean. No obvious errors. Professional charts. All the statistics you want. The leakage is invisible unless you know to look for it.

Step 3: The Core Mechanism

Purged K-fold adds two operations to standard K-fold:

Purging removes training samples whose labels (future returns) overlap with the test period. If your test set is days 801-850, you remove training samples from the gap around the boundary — typically a few days before and after the test set.

Embargo adds a gap after each test set. The next training period doesn’t start until after the embargo period ends. This accounts for the persistence of information effects.

Here’s the picture:

Fold 1:
  Train: days 1-800
  Purge: days 801-(800+n_purge)  ← removed from training
  Test:  days (801+n_purge)-(850+n_purge)
  Embargo: days (851+n_purge)-(850+n_embargo)  ← removed from both

Fold 2:
  Train: days (embargo_end)-end
  ...

The purge gap prevents information at the boundary from leaking. The embargo gap prevents serial correlation in the returns from inflating performance.

Step 4: The Key Formulas

Embargo size is typically a percentage of total data: $n_{embargo} = \lfloor T \times pct_{embargo} \rfloor$

For 5 years of daily data (T ≈ 1,260) and pct_embargo = 0.01: n_embargo = 12 days.

Number of test samples per fold: $n_{test} = \left\lfloor \frac{T - n_{embargo}}{k} \right\rfloor$

For T=1260, n_embargo=12, k=5: n_test = 249 days.

Effective training size for fold i: $n_{train,i} = T - (i+1) \cdot n_{test} - n_{embargo} - n_{purge}$

OOS Sharpe (the right metric): $\widehat{SR}_{OOS} = \frac{\hat{\mu}_{OOS}}{\hat{\sigma}_{OOS}} \cdot \sqrt{252}$

Where $\hat{\mu}_{OOS}$ and $\hat{\sigma}_{OOS}$ are the mean and std of out-of-sample returns across all folds.

Step 5: Implementation

import numpy as np
import pandas as pd
from sklearn.base import clone
from sklearn.model_selection import KFold

def purged_kfold(X, y, k=5, pct_embargo=0.01, n_purge=None):
    """
    Purged K-fold for financial time series.

    Parameters
    ----------
    X : array (n_samples, n_features)
    y : array (n_samples,)
    k : int
        Number of folds
    pct_embargo : float
        Embargo as fraction of total samples
    n_purge : int, optional
        Purge size. Defaults to sqrt(n_samples)

    Yields
    ------
    train_indices, test_indices : arrays
    """
    n = len(X)
    n_embargo = int(n * pct_embargo)
    if n_purge is None:
        n_purge = int(np.sqrt(n))

    n_per_fold = (n - n_embargo) // k

    for i in range(k):
        # Test window
        test_start = i * n_per_fold + n_purge
        test_end = test_start + n_per_fold

        # Training window (before purge gap)
        train_end = test_start - n_purge
        train_start = 0

        # Embargo after test window
        embargo_end = min(test_end + n_embargo, n)

        train_idx = np.arange(train_start, train_end)
        test_idx = np.arange(test_start, test_end)

        yield train_idx, test_idx


def evaluate_strategy(model, X, y, k=5, pct_embargo=0.01):
    """
    Evaluate a strategy using purged K-fold CV.

    Returns OOS Sharpe, OOS returns, and fold-by-fold breakdown.
    """
    oos_returns = np.zeros(len(y))
    fold_returns = []

    for train_idx, test_idx in purged_kfold(X, y, k=k, pct_embargo=pct_embargo):
        X_train, X_test = X[train_idx], X[test_idx]
        y_train = y[train_idx]

        model_clone = clone(model)
        model_clone.fit(X_train, y_train)
        preds = model_clone.predict(X_test)

        # Assume preds are positions; returns = preds * actual_return
        fold_ret = preds * y_test if 'y_test' in dir() else preds * y[test_idx]
        oos_returns[test_idx] = fold_ret
        fold_returns.append(fold_ret)

        y_test = y[test_idx]  # Store for next iteration

    # Overall OOS Sharpe
    oos_sharpe = np.mean(oos_returns) / np.std(oos_returns) * np.sqrt(252)

    return {
        'oos_sharpe': oos_sharpe,
        'oos_returns': oos_returns[oos_returns != 0],
        'fold_returns': fold_returns,
        'fold_sharpes': [np.mean(fr) / np.std(fr) * np.sqrt(252) for fr in fold_returns]
    }

Step 6: How to Read Results

The OOS Sharpe is what matters. Not the in-sample Sharpe. Not the full-sample Sharpe. The Sharpe computed only on the held-out test sets across all folds.

A common mistake: reporting Sharpe from the combined OOS predictions, which is correct. But also looking at the full-sample Sharpe “for reference” — this is misleading because it includes data that was used for training.

Look at the fold-by-fold breakdown:

• If Fold 1 Sharpe is 2.1 and Fold 2 Sharpe is -0.8, your strategy is regime-dependent. It’s not robust.
• The average matters less than the variance. A strategy with average Sharpe 0.8 and fold SD 0.3 is more robust than one with average 1.2 and fold SD 1.5.

Report the minimum fold Sharpe. If your worst fold still has Sharpe > 0.5, the strategy is robust. If your worst fold is negative, be suspicious.

Step 7: Common Pitfalls + Where It Fits

Pitfall: Using pct_embargo=0. This is the most common mistake. If you’re not embargoing, you’re not doing time-series CV. The cost of embargo is efficiency — you use less data. The benefit is honest estimates. Always use some embargo, even 0.5%.

Pitfall: Purge size too small. n_purge = sqrt(n) is a rule of thumb. But if your signal has slow decay (e.g., a 200-day moving average), sqrt(n) might be too small. Calibrate to your signal’s effective half-life.

Pitfall: Comparing CV to full-sample. Your purged K-fold Sharpe will be LOWER than your full-sample Sharpe. This is correct and expected. If someone shows you a backtest with a 2.1 Sharpe and says “and we use purged CV,” but their purged CV Sharpe is also 2.1, they’re not actually purging.

Where purged K-fold fits:

• Validating any strategy that trains on historical data to predict future returns
• Comparing strategy variants (use the same purge/embargo settings for fair comparison)
• As input to more advanced methods (PBO uses CPCV, which extends purged K-fold)

Purged K-fold is the foundation. Combinatorial purged CV (CPCV) extends it. Deflated Sharpe uses OOS returns from purged CV. PBO uses CPCV. If you’re doing anything with backtests, you need purged K-fold first.

Next Steps

Ready for the next level? → Walkthrough 02: Combinatorial Purged CV

Want the encyclopedia reference card? → Time Series CV (Encyclopedia)