Backtesting & Validation
8 articles ~1.5 h
- Cross-Validation Methods for Quant Strategies: K-Fold vs Walk-Forward vs Purged vs CPCV There are at least four ways to do cross-validation in time series: standard k-fold, walk-forward, purged CV, and combinatorial purged CV. Each has different bias-variance trade-offs.
- Live vs Backtest Performance: The Gap You Should Plan For Live performance is always worse than backtest performance. The gap depends on strategy, market regime, and execution, and it should be estimated before deployment.
- Multiple Testing and the Deflated Sharpe Ratio If you test 100 strategies, you'll find winners by chance. The Deflated Sharpe Ratio adjusts the Sharpe for the number of trials and the distribution of returns. Without it, your Sharpe ratio is fiction.
- Probability of Backtest Overfitting (PBO): How to Know Your Best Strategy Is Probably the Worst The Probability of Backtest Overfitting (PBO) uses CSCV to estimate how often an in-sample-selected winner ranks below the out-of-sample median.
- Stress Testing and Regime Robustness: Will Your Strategy Survive 2027? A strategy that works in 2010-2019 might collapse in 2022. Stress testing across regimes, crashes, and synthetic perturbations is the last line of defense before deployment.
- Walk-Forward Analysis Walk-forward is the minimum honest simulation of live trading: train on the past, test on the next period, roll forward. What makes it valuable is that it hands you a distribution, not a number.
- What Is Backtesting and What Is Validation A backtest is a calculation; validation is the hypothesis test on that calculation. What each step of the five-step pipeline actually proves, and how strong a claim its number will support.
- The White Reality Check and the SPA Test The White Reality Check and the SPA test answer a specific question: is my best strategy's performance statistically distinguishable from what random data snooping could produce?