Synthetic Data Generation: The Ornstein-Uhlenbeck Backtest

Your Backtest Has Seen Exactly One Path

A historical backtest carries a bias that is easy to miss: you are testing the strategy on the one path the market happened to take. The strategy making money on that path does not mean it makes money on a different one. The market could have taken another path with identical statistical properties, and your strategy could have lost on it.

This is not a philosophical objection. Your backtest reports a Sharpe — say 1.6. How uncertain is that number? A single-path backtest cannot tell you, because it has exactly one sample point.

The fix is direct: generate a batch of synthetic paths that share the statistical properties of the history, and run the same strategy on each one. What you get back is not a Sharpe but a distribution of Sharpes. The mean is your estimate of expected performance; the width is how much confidence that estimate deserves. That is a far more honest object than a single number.

The reference for this is AFML, Chapter 13 (“Backtesting on Synthetic Data”), and the basic building block is the Ornstein-Uhlenbeck process.

(López de Prado, M., Advances in Financial Machine Learning, Ch. 13. ISBN 978-1119482086.)

The OU Process: A Block You Can Copy Freely

OU is a mean-reverting stochastic process:

where $\kappa$ is the mean-reversion speed, $\mu$ is the long-run mean, $\sigma$ is the volatility, and $W_t$ is a standard Brownian motion. The drift term $\kappa(\mu - X_t)$ is the whole mechanism: the further the process wanders, the harder it gets pulled back.

Discretized with Euler-Maruyama, in a form you can drop straight into a loop:

where $\epsilon_t \sim N(0, 1)$.

OU is stationary: constant mean $\mu$, constant variance $\sigma^2 / (2\kappa)$, and autocorrelation $e^{-\kappa \tau}$ at lag $\tau$, decaying exponentially. Those three properties decide where it applies. Spreads, basis, volatility, term structure — quantities that have a level they belong at. Prices do not. Force OU onto a trending quantity and you will generate a pile of fake paths circling the historical mean, then conclude that your mean-reversion strategy makes money on every path. That conclusion is manufactured entirely by your model specification. Trending quantities want geometric Brownian motion or another process with drift.

(Uhlenbeck, G. E., & Ornstein, L. S., “On the Theory of the Brownian Motion,” Physical Review, 36(5), 1930, pp. 823-841.)

Calibration, and the Step That Is Usually Wrong

Given a historical series, you need $\kappa, \mu, \sigma$. The long-run mean is the sample mean:

The reversion speed comes from the lag-1 sample autocorrelation $\hat{\rho}$. Observed discretely, OU is an AR(1) whose autoregressive coefficient equals $e^{-\kappa \Delta t}$, so inverting gives:

with one precondition: $\hat{\rho}$ has to be positive. If your series has negative or near-zero lag-1 autocorrelation, the log is either undefined or blows up into an enormous $\kappa$. That is the data telling you it does not look like OU, and the right response is a different model, not a forced calculation.

The volatility step is where implementations go wrong. $\sigma$ must be estimated from the AR(1) residuals, not from the increments:

where $\hat{s}_\varepsilon^2$ is the sample variance of $\hat{\varepsilon}_t = (X_{t+\Delta t} - \hat{\mu}) - \hat{\rho}(X_t - \hat{\mu})$. The denominator $1 - e^{-2\kappa\Delta t}$ is precisely the factor that converts residual variance back into $\sigma^2$, and it is only valid when paired with the residual. Put the variance of the first differences $X_{t+\Delta t} - X_t$ in the numerator instead and you are double-counting the displacement that mean reversion pulled back, so $\hat{\sigma}$ comes out systematically too large — and the bias grows with $\kappa$. This mistake is common because “variance of the increments” sounds obviously right.

A minimal calibration in Python:

import numpy as np

def calibrate_ou(series, dt=1.0):
    """Calibrate OU parameters from a time series."""
    mu = series.mean()
    rho = np.corrcoef(series[:-1], series[1:])[0, 1]
    kappa = -np.log(rho) / dt
    resid = (series[1:] - mu) - rho * (series[:-1] - mu)
    s_eps_sq = np.var(resid, ddof=1)
    sigma_sq = s_eps_sq * 2 * kappa / (1 - np.exp(-2 * kappa * dt))
    sigma = np.sqrt(sigma_sq)
    return {'mu': mu, 'kappa': kappa, 'sigma': sigma}

For scale: a daily series with lag-1 autocorrelation of 0.95 gives $\kappa \approx 0.051$ per day, or roughly 13 per year at 252 trading days. The half-life is $\log 2 / \kappa \approx 13.5$ days — a deviation decays halfway back in about two and a half weeks. Check that number against the raw series before trusting it. If it does not match what you see, the calibration window probably straddles two different regimes.

What a Thousand Paths Looks Like

With parameters in hand, generation is the discretization in a loop:

def generate_ou_path(mu, kappa, sigma, x0, n_steps, dt=1.0, seed=None):
    """Generate one OU path."""
    rng = np.random.default_rng(seed)
    path = np.zeros(n_steps)
    path[0] = x0
    for t in range(1, n_steps):
        path[t] = path[t-1] + kappa * (mu - path[t-1]) * dt + sigma * np.sqrt(dt) * rng.standard_normal()
    return path

Call it repeatedly with different seeds and you get a family of paths: same mean, variance, and autocorrelation, different realizations. For a backtest you typically want 1000 or more, running the strategy on each and collecting the Sharpe. Ten paths will also produce a histogram, but the shape of that histogram is mostly sampling noise rather than a property of the strategy.

The AFML framework in Chapter 13 goes past this single block. It aims to match the historical data on three levels at once: the marginal distribution should share the same mean, variance, skewness, and kurtosis; the serial dependence should share the same autocorrelation structure; and for multi-asset data, the cross-sectional dependence should share the same correlation structure. The mechanism is to calibrate the marginals and the dependence structure separately and then glue them back together with a copula — marginals typically fitted to the empirical distribution via kernel density estimation, dependence via the empirical correlation matrix.

If you would rather not implement a copula, there is a simplification that is usually enough:

1. Fit a multivariate normal to the historical data, after transforming the marginals via the empirical CDF.
2. Generate synthetic multivariate normal samples with the same covariance matrix.
3. Transform back to the original marginals via the inverse empirical CDF.

The output matches the historical marginals and correlation structure. The cost is that it collapses dependence into linear correlation, which throws away tail dependence — a shortcut that comes back to bite later.

Reading the Historical Number Against the Distribution

Once you have a distribution, the lone Sharpe from the historical backtest stops being a conclusion and becomes a point on that distribution. The rigorous version is to run both and compare positions: get the Sharpe estimate from the historical backtest, get the Sharpe distribution from 1000 synthetic paths, and see where the historical value lands.

Inside the bulk of the distribution means the strategy is doing about what its statistical properties permit, with no visible path-specific overfitting. In the upper tail admits two readings — it may be overfit to that particular path, or it may have captured a genuine edge the synthetic model does not represent — and separating those requires the tool in the next section. Below the median means either bad luck or a strategy that fails to exploit these statistical properties and is being dragged down by its own rules.

This is the actual problem synthetic backtesting solves. The bias of “one path” is replaced by the uncertainty of “many paths,” and uncertainty is at least something you can quantify. The historical backtest hands you a point estimate with no error bar; the synthetic backtest hands you the error bar.

There is also a cheap visual diagnostic: plot the historical equity curve on the same axes as a dozen synthetic ones. If the historical curve disappears into the crowd, the strategy is robust. If it is instantly the prettiest line on the chart, suspect overfitting. This catches shape anomalies that summary statistics miss — a historical curve that is implausibly smooth, for instance.

(AFML, Ch. 13.)

Deflate Any Sharpe You Searched For

All of the above can be defeated in one move: you backtest 100 strategies on synthetic data, take the one with Sharpe 2.5, and declare it good. That conclusion ignores multiple testing — among 100 strategies with no edge whatsoever, a few should post high Sharpes by luck alone. Synthetic data grants no exemption here. It only moves the search to a different arena.

The Deflated Sharpe Ratio (DSR) exists for this. It folds in multiple testing, non-normality of returns, and track-record length, and it is the correct metric for a strategy that was found by searching:

where $\widehat{SR}$ is the observed Sharpe at the same frequency as $T$ (not annualized), $T$ is the number of return observations, $\hat{\gamma}_3$ is skewness, $\hat{\gamma}_4$ is kurtosis, and $Z[\cdot]$ is the standard normal CDF. The skewness and kurtosis terms in the denominator do real work: negative skew and fat tails inflate the variance of the Sharpe estimate itself, so the same $\widehat{SR}$ is weaker evidence when returns are fat-tailed.

The threshold $SR_0$ is not zero. It is the expected maximum Sharpe that pure noise reaches across $N$ trials:

where $V$ is the cross-trial standard deviation of the Sharpe estimates and $\gamma \approx 0.5772$ is the Euler-Mascheroni constant. $N$ enters logarithmically, so trying 100 variants and trying 10,000 sit within a small factor of each other — but both sit far above trying one. Read the DSR as the probability that the true Sharpe exceeds that threshold once multiple testing is accounted for. Above 0.95 is where the strategy starts to stand up.

The load-bearing input is $N$, and it has to be honest. You know how many variants you tried. Enter 1 and the formula corrects nothing.

(Bailey, D. H., & López de Prado, M., “The Deflated Sharpe Ratio,” Journal of Portfolio Management, 40(5), 2014.)

A Worked Example

The setup: a mean-reversion strategy on the spread between two cointegrated assets. The spread is approximately OU with $\mu = 0$, $\kappa = 0.05$, $\sigma = 1$. Historical data covers 5 years, 1250 daily observations.

Calibrating to the historical spread gives $\hat{\mu} = 0.02$ (slightly positive), $\hat{\kappa} = 0.048$ (about 12 per year), and $\hat{\sigma} = 1.05$. From those parameters, generate 1000 paths, each 1250 days long, identical parameters and different random seeds.

The strategy rule is simple: enter when the spread is 2 standard deviations from the mean, exit when it reverts. Run it on every synthetic path and record the Sharpe.

The resulting distribution is roughly normal with mean 1.3 and standard deviation 0.4; the normal quantiles put the 5th percentile near 0.64 and the 95th near 1.96. That alone is informative. Even with the strategy, the parameters, and the statistical properties all held fixed, the Sharpe swings between roughly 0.64 and 1.96 purely because the path differs. Any backtest that reports a Sharpe to two decimal places is fooling itself against that width.

The historical backtest gave 1.6, which lands around the 77th percentile of the distribution — on the high side, nowhere near anomalous. The DSR works out to 0.92, below the 0.95 line, meaning the strategy is probably genuine but with residual uncertainty that has not been ruled out.

Put together: expected Sharpe around 1.3, with what you actually get depending heavily on which path the market takes, and a historical 1.6 that is consistent with that range and needs no overfitting to explain. That reads as less impressive than “backtest Sharpe of 1.6” — and it carries considerably more information.

(AFML, Ch. 13.)

The Same Lever, Pulled the Other Way

You choose the parameters that generate the paths, which means you can choose parameters that history never produced. This is the second use of synthetic data beyond uncertainty quantification, and it is often the more valuable one. Historical data cannot answer “what if volatility doubles,” because the sample may contain no such period.

Continuing the example above, push in three directions. Set $\sigma = 2$ against the calibrated 1.05 to simulate a volatile regime and see whether the strategy still functions. Set $\kappa = 0.01$ against the calibrated 0.048 to simulate materially weaker mean reversion and see whether the edge survives. Set $\mu = 1$ against the calibrated 0.02 to simulate a spread with a persistent bias — usually the most lethal of the three, because a strategy built on reversion to a mean will keep adding to a losing position while the mean itself drifts.

Generate 100 or more paths per stress scenario and evaluate. A common result: a strategy that performs well in the calibrated regime fails outright under stress. That is close to a definition of overfitting to the historical regime. A robust strategy does not have to make money in stressed regimes, but it should keep a positive Sharpe across most of them, or at minimum lose in a bounded and predictable way.

Where the Model Lies to You

The honesty of synthetic data has boundaries, and your choice of model decides where those boundaries sit.

The most fundamental: synthetic paths are not the future. They are generated from calibrated statistical properties, so if the future regime differs from the historical one, the paths do not apply. A synthetic backtest tells you how the strategy would have done across plausible alternative histories, not how it will do going forward. The gap between those two statements is exactly how strong an assumption you are making about the stability of market structure.

Tails are the second hole, and this one is specific to OU. OU increments are Gaussian while real financial returns are fat-tailed, with extreme events arriving more often than a Gaussian predicts. Synthetic OU paths therefore systematically under-represent tail events, so any strategy that lives or dies on tail behavior — option selling being the canonical case — gets badly mis-evaluated, and mis-evaluated in the flattering direction. The empirical-CDF variant described earlier recovers some of the marginal fat tail, but tail dependence across assets, the phenomenon where everything falls together in a crisis, is not something a linear correlation matrix can capture.

Third, path-dependent strategies are hard to evaluate. Drawdown-based position sizing, hold-until-target exits, and rules of that kind depend on the specific ordering of events. Two paths with identical statistical properties can have very different drawdown sequences, and therefore different outcomes. This is not really a defect — the width of the synthetic distribution is supposed to absorb that variability — but it does mean the distribution for such strategies is wider than it looks, and you need more paths before the quantiles settle down.

Finally, cross-asset dependence shifts. The historical correlation matrix may not hold in the future, and synthetic data preserves it faithfully. What you preserved may be precisely the thing that is about to break.

Choosing the wrong model amplifies all of the above at once. OU suits mean-reverting quantities. Trending quantities want a random walk or geometric Brownian motion, volatility wants GARCH, and multi-asset data wants a multivariate model with an appropriate dependence structure. Generate stock price paths with OU and then discover that your mean-reversion strategy is profitable, and what you tested was your model, not your strategy.

(AFML, Ch. 13.)

When the Quantity Is Not Mean-Reverting: Markov Chains

For discrete signals — order book states, regime labels, bucketed signal levels — OU does not apply at all; a continuous process has no meaning on discrete states. Use a Markov chain instead.

A Markov chain is defined by a transition matrix $P$, where $P_{ij}$ is the probability of moving from state $i$ to state $j$ in one time step. Calibration is three steps:

1. Encode the historical signal as a sequence of states.
2. Count transitions: $n_{ij}$ = the number of times state $i$ is followed by state $j$.
3. Estimate $P_{ij} = n_{ij} / \sum_k n_{ik}$.

To generate, start from the empirical distribution of states and transition according to $P$ at each step. What survives is what a discrete signal ought to preserve: the stationary distribution, the transition probabilities, the autocorrelation structure. For multi-state labels such as bull, bear, and sideways, the chain also produces regime switches at the correct rates — something OU cannot do, because OU has exactly one regime.

The limitation is order. A first-order chain conditions only on the previous state, so long-memory dependence is invisible to it; if your signal has structure like “three consecutive moves in the same direction make a fourth more likely,” a first-order chain flattens it out. Longer memory means higher-order chains (order 2, 3, and so on) or hidden Markov models, at the cost of a parameter count that grows exponentially in the order, which exhausts your sample quickly.

(Hidden Markov Models for regime detection are discussed in Hamilton, J. D., “Analysis of Time Series Subject to Changes in Regime,” Journal of Econometrics, 53(1-3), 1990, pp. 7-33.)

The Block Bootstrap: The Cheap Route

Both OU and Markov chains ask you to commit to a model first. The block bootstrap does not:

1. Extract overlapping blocks of length $L$ from the historical series.
2. Resample the blocks with replacement.
3. Concatenate the resampled blocks into a synthetic series.

Within-block structure — autocorrelation, distributional properties — is preserved; between-block structure is destroyed. The synthetic series is essentially a random rearrangement of historical fragments. At $L = 21$, one trading month, monthly-scale patterns survive and yearly ones do not; what comes out looks like a year assembled from randomly drawn historical months.

The appeal is that it takes a few lines, requires no distributional assumptions, and carries skewness and kurtosis straight over from the data rather than flattening them under a Gaussian assumption. The fat-tail problem essentially does not arise on this route.

The costs are equally clear. The synthetic series is bounded by the historical blocks and cannot produce new patterns — a combination that never occurred in history cannot be bootstrapped into existence. The seams between blocks introduce unrealistic discontinuities, and the smaller $L$ is, the more seams there are. Long-memory structure cannot cross a block boundary, so like a first-order Markov chain, it loses long-range dependence.

That makes it well suited to quick sanity checks and less rigorous than the model-based routes. Use it as a complement, not a substitute.

(Künsch, H. R., “The Jackknife and the Bootstrap for General Stationary Observations,” Annals of Statistics, 17(3), 1989, pp. 1217-1241.)

Disagreement Between Methods Is the Informative Case

With all three routes on the table, the sound approach is to walk all three: 1000 paths from OU, 1000 from the block bootstrap, 1000 from a Markov chain, then compare the three Sharpe distributions.

Similar shapes mean the strategy’s performance does not hinge on the choice of synthetic method, and the conclusion is safe to use. Different shapes mean the conclusion actually depends on a modeling choice, and the productive response is to work out which assumption is driving it — usually tails, long memory, or regime switching.

One pattern shows up almost every time, so it is worth knowing in advance. OU produces the tightest Sharpe distribution, because OU paths are smoother than bootstrap or Markov paths. The block bootstrap produces the widest, because it can stitch together non-adjacent unusual periods and manufacture sequences uglier than anything real history contains.

That difference is not a matter of one being right. It is a matter of purpose. For risk management, take the wider distribution from the block bootstrap, since what you want to know is how bad the bad case gets. For estimating expected performance, the tighter OU distribution may be more accurate, since it does not include discontinuities that are artifacts of stitching. Look at both and let the use case decide which one you weight.

What It Buys

What you get is a more honest estimate of performance. The mean Sharpe across synthetic paths is closer to expected performance than the historical Sharpe, and the standard deviation across paths hands you the uncertainty directly.

Expect the estimate to move down. The historical Sharpe was computed on a path you have looked at repeatedly, and the synthetic mean will land noticeably below it — a haircut of a quarter to a half is a common order of magnitude, though the size varies a great deal with the strategy, the sample length, and how many variants you searched, so it should never be applied as a fixed discount. In the example above, a historical 2.0 corresponding to a synthetic mean of 1.3 with a standard deviation of 0.5 is entirely normal: the 2.0 was optimistic, 1.3 is more realistic, and that 0.5 tells you not to treat even 1.3 as settled.

The cost is engineering. Generating synthetic data, running a backtest on every path, and analyzing the distribution takes both code and compute, and 1000 paths of 1250 days is not something you run casually. For simple strategies with few parameters and transparent logic, the historical backtest plus some common sense may be enough. For complex strategies with many hyperparameters, heavily searched, where you cannot quite articulate why this parameter set won — that is where synthetic backtesting pays best, because that is precisely the situation most likely to have fooled you.

In the wider evaluation chain, synthetic backtesting is one link. The historical backtest is the baseline, showing what would have happened on the path that actually occurred. The synthetic backtest supplies plausible alternative histories and turns the conclusion from a point into a distribution. Stress testing pushes parameters to extremes to find where the strategy breaks. Walk-forward optimization refits and re-evaluates on rolling windows, testing whether the strategy keeps up with changing conditions. Paper trading is the only link that exposes slippage, fills, and latency, since all four of the others assume you transact at theoretical prices.

Each method has its own blind spot and each covers the previous one’s, which is why a rigorous evaluation runs all five. A strategy that passes all of them is a candidate for live deployment. A strategy that fails one is a reason to go find out why, not a reason to route around it.

(López de Prado, M., Advances in Financial Machine Learning, Ch. 13. ISBN 978-1119482086.)