ML Trading Strategy Lifecycle

Most ML research dies between the Jupyter notebook and the live trading account. Not because the model was wrong, but because nobody planned the process that gets it deployed.

Compress the whole lifecycle into one number and that number is a ratio: actual over backtested. Research produces a Sharpe. Paper trading discounts it. The pilot discounts it again. Scaling up discounts it again. Then time keeps discounting it year after year until what remains no longer justifies the capital. Every gate asks the same question: is this round’s discount still inside what you predicted? Strategies do not die because a model suddenly stops working. They die because one gate’s discount came in far worse than expected and nobody was watching that ratio.

This article walks the six stages: research, paper trading, pilot, full deployment, monitoring, and retraining or retirement. How long each should take, what passing looks like, and which ratio tells you to stop.

Scope first. This is a pipeline sized for a typical small-team quant operation. Institutions with full compliance, risk, and ops functions add gates and move slower. Same principles, different velocity.

Stage 1: Research (Weeks to Months)

Research produces a strategy you believe in but have not yet trusted with money. Inputs are data, a hypothesis, and some engineering. Outputs are backtested performance with honest statistics, walk-forward validation, and a Deflated Sharpe Ratio you can defend.

The first half is routine: build the factor or ML model (for ML strategies that means feature engineering, model selection, walk-forward cross-validation, and hyperparameter tuning with proper out-of-sample testing), then compute the standard backtest statistics — Sharpe, Sortino, max drawdown, Calmar, turnover, capacity estimate, factor exposures. Nobody argues about those numbers. The arguments are about the discounts applied to them next.

The first discount is the Deflated Sharpe Ratio. DSR adjusts the reported Sharpe for the number of strategies you tried, the length of the backtest, and the non-normality (skewness and kurtosis) of the return distribution. One thing to be precise about: AFML’s DSR (§14.7.3) does not directly adjust for return autocorrelation, and for autocorrelated returns you need a separate effective-sample correction. The magnitude of the discount is startling — a naive Sharpe of 1.5 over ten years of daily data often comes down to 0.6 or 0.7 after adjustment.

The second is the Probability of Backtest Overfitting. PBO estimates the probability that the in-sample-optimal configuration underperforms the median out of sample. By that definition 50% is a coin flip, and many teams draw the line at 25%: above it, assume you have fit noise rather than signal.

The third is costs. Transaction costs, slippage, and capacity drag typically knock a further 50-70% off the backtest Sharpe. A backtest Sharpe of 1.0 in research generally corresponds to an expected live Sharpe of 0.3-0.5.

That is where the honesty gate sits. If, after all these adjustments, you cannot honestly claim an expected live Sharpe above 0.5, do not proceed. Lower than that is not a deployable strategy, it is an experiment, and the only open question is whether it loses money slowly or quickly.

Put those last two numbers next to each other for a second. A strategy with a backtest Sharpe of 1.0 lands on or under the gate once adjusted. To have any headroom you need a backtest Sharpe north of 1.5 — which is precisely the region where DSR cuts hardest. Those two constraints together are the real difficulty of the research stage.

This stage takes weeks to months for a competent team. Don’t rush it. The cost of a slow research stage is opportunity cost; the cost of a sloppy one is next year’s entire strategy budget.

Stage 2: Paper Trading (3-6 Months Minimum)

Paper trading runs the strategy against live market data without committing capital. The order management system generates hypothetical trades, portfolio accounting simulates fills, costs, and PnL, and nothing real is at risk.

What it validates is not whether the strategy is right — that was the previous stage — but whether the live data pipeline, signal generation, portfolio construction, and risk management all behave as designed when the thing is actually running. A backtest is an offline simulation. Paper trading is the same system running in production, with the single difference that the money is fake.

The first class of problem it catches is latency and timing. Is your signal generated close enough to the price bar you are using? Are you computing features correctly on live data? Mistakes here are embarrassingly common and invisible in a backtest. The second is the data feed: does your vendor’s live feed match the history you backtested on? Adjustment conventions, survivorship, and timestamp semantics all surface here. The third is operational reliability — whether the system runs for weeks without crashing, and whether it survives exchange halts, bad ticks, and network blips. The fourth is fills: some backtests use yesterday’s close as the fill price, while live markets use today’s, plus slippage.

On the ratio, paper Sharpe usually lands at 50-70% of backtest Sharpe. A fully systematic daily strategy that never trades intraday can run 60-80%, because it depends less on execution detail in the first place. Below 30% means the strategy is broken, and there are only three usual causes: live data does not match backtest data, transaction costs were understated, or there is lookahead in the backtest that you have not found yet.

As for why the minimum is 3-6 months: a one- or two-month window is often entirely calm market and never stresses the system. You want at least one regime shift, one volatility spike, and one stretch of unusual price action inside the window, and then you want to see the strategy survive them.

Healthy at the end of the window, go to pilot. Not healthy, fix the system or kill the strategy.

Stage 3: Pilot Deployment (3-6 Months)

The pilot commits a small fraction of intended capital, typically 5-10%. The strategy runs in production with real orders, real fills, real slippage, and real counterparty risk, on an amount small enough that total failure is a survivable loss.

Paper trading validated the system; the pilot validates the market’s reaction to you. This is the first time somebody is on the other side of your orders, and the first time you get real numbers for five things. Realized slippage: what you assumed in the backtest versus what production actually gives you, small for liquid US equities and substantial for thinner names or larger orders. Capacity: how much capital the strategy can carry before its own footprint moves prices, given that something working on $1M may not work on $100M. Fill rate: the fraction of intended trades that actually execute, where 95% is fine and 60% means your signal is saying “I’d like to trade, but only at a price that doesn’t exist.” Operational costs: the team hours, vendor fees, and infrastructure that backtests omit. And the first direct evidence of capacity drag.

Pilot Sharpe typically comes in at 40-60% of backtest Sharpe. Note that this band sits below the paper band, and the extra discount is exactly the price of slippage, capacity, and fill rate — none of which paper trading can see. Below 40% means there is a problem you have not located yet. Landing within 80% of backtest Sharpe is excellent and usually means liquid names, clean execution, and a correctly specified cost model.

Evaluation runs 3-6 months for the same reason as paper trading: you want to cover at least one regime. Healthy pilot, move to full deployment. Unhealthy, cut the allocation, restructure, or kill it. Don’t push a bad pilot forward hoping it improves on its own.

Stage 4: Full Deployment (Gradual Scaling)

Full deployment does not mean “deploy everything tomorrow.” It means scaling gradually under monitoring, and a common pattern is 30% of intended capital in month one, 50% in month two, 75% in month three, 100% in month four.

The reason for four steps is that capacity drag only reveals itself at size. The portion of returns eroded as capital grows often stays invisible until the strategy manages more than half of intended capital. Rushing to full size makes your first hint of capacity drag and your first large loss the same event.

At this stage the question shifts from “does it run” to “is it running right.” Live Sharpe gets compared against expected on daily, weekly, and monthly rolling windows. Factor IC — the rank correlation between your signal and realized returns — gets measured live, and most strategies watch it decline year over year as the alpha gets crowded. Turnover typically runs higher in production than in backtest, especially once liquidity-constrained assets start misbehaving. Realized transaction costs get compared against modeled costs, and any understatement gets amplified by turnover on its way out of your Sharpe, which makes the leverage brutal for high-turnover strategies. Risk limits are the one item requiring no judgment at all: position concentration, leverage, sector exposure, and drawdown caps are enforced automatically, with no discretionary escape hatch.

Full deployment never “finishes.” It is a continuous state of monitoring and adjustment, and the strategy either stays in it until it dies or the team deliberately retires it.

Stage 5: Monitoring and Decay Detection

Every strategy decays. Half-lives vary widely — value strategies decayed through the 2010s, momentum has multi-decade cycles, and whether mean reversion persists depends on the regime. Some strategies last six months, some last ten years, none last forever.

Monitoring is just the same ratio measured continuously, with the denominator switched from “backtest” to “what research expected.”

Live IC is the most sensitive of them. Compute the rolling rank correlation of signal and realized returns, usually over 60-120 day windows, and compare it to the expected IC from research. Above 80% means the strategy is working. 30-50% means significant decay. Below 30% means the strategy is dead and the capital should come out.

Realized Sharpe is the same logic applied to actual returns, only much slower — a daily strategy needs 12-18 months live before it has statistical power, and before that a mismatch is quite likely just small-sample noise. Which is why IC is your early warning and Sharpe is your confirmation. Don’t run them in the other order.

Capacity utilization tells you which part of the curve you are reading. If the strategy’s natural capacity is $500M and you are running $50M, you are at 10% capacity: drag is small, but your live PnL also does not show the strategy’s true Sharpe. At 80% capacity drag starts to bite, and any decline you observe has to be decomposed into genuine decay versus the size you added yourself.

Drawdown is the hard boundary. A drawdown at 1.5x the expected max from research suggests a regime shift and warrants a review of risk limits. At 2x the expected max, most teams pause the strategy and go back over it.

All of this needs explicit alerting thresholds, and listing them out is worth it because they end up as literal lines in a config file:

• IC drops 30% below its rolling 6-month average → alert.
• Live Sharpe drops below 0.3 over a rolling 12-month window → alert.
• Drawdown exceeds the expected max by 50% → alert and pause.
• Fill rate drops below 80% of intended → alert.
• Operational errors (data outages, missing fills, system crashes) → alert.

Automate the alerts. Manual monitoring at the frequency required to catch IC decay is not something a human can do.

Stage 6: Retraining

Monitoring fires an alert and the first response is retraining: refit on recent data (rolling window, typically 3-5 years), add new features as market dynamics evolve (new macro indicators, alternative data), and re-validate walk-forward on the most recent period.

If retraining restores IC and Sharpe to expected levels, the strategy lives. If it doesn’t, you have one of three problems, and they demand completely different responses.

The alpha may be structurally gone: the market opportunity that produced it has been arbitraged away and no amount of retraining brings it back. The model may be mis-specified: the relationship between features and returns has changed, so what you need is new features rather than a refit, and retraining here just hands you a well-fit model with no predictive power. Or it may be noise masquerading as decay: IC dropping for a quarter and recovering is ordinary, and killing a healthy strategy over three bad months is a real way to lose.

Telling the three apart takes time and careful analysis, and it takes a willingness to admit when the answer is “the alpha is gone.” Teams reach that verdict too rarely rather than too often, and the reason is hope. Be honest about what the data says.

Stage 6 (Alternative): Retire

Some strategies do not recover. IC stays low, Sharpe stays low, and retraining with new features does not help. That strategy is done.

Retirement has an order to it. Stop allocating capital, pause or shut down the production system, then run down the remaining positions in an orderly way — do not dump everything at once, which manufactures exactly the market impact you spent the whole lifecycle avoiding. The last step is documenting what worked, what didn’t, and what you learned for next time. That step is the one most often skipped, and it is the only part of a retirement that compounds.

A retired strategy is not a failure. A retired strategy that taught the team how to build the next one is a success. Running a decayed strategy because nobody will admit it is dead is how teams lose money they cannot afford to lose.

The Honest Decay Timeline

Lay that ratio out year by year and the realistic rhythm for most ML-driven strategies looks like this. Year one is full alpha and peak Sharpe, with the signal still uncrowded. Year two retains 70-90% of peak as other participants start to notice and natural decay sets in. Year three retains 50-70%, at which point crowding is real: either you grew too large, or others copied it, and usually both. Year four and beyond leaves 30-50% if you are lucky, and absent a fundamental redesign, most strategies are retired by year three or four.

The numbers vary by strategy, and the dividing lines are basically turnover and how crowded the instrument already is. A long-short value equity strategy turning over 20% a year decays slowly; high-frequency stat-arb turning over 1000% a year decays quickly. Statistical patterns in less-traded instruments — corporate bonds, commodities, esoteric futures — persist longer than patterns in heavily-traded ones like S&P 500 constituents and major FX pairs, because fewer people are looking.

So plan around the decay and don’t bet a career on one strategy lasting five-plus years. Keep replacement strategies in the research pipeline, and keep at least one strategy in pilot at all times, because the day your main strategy dies you need something you can scale up rather than three years of stages to walk from the top.

The Six Recurring Killers

Run a few of these lifecycles and the ways strategies die start repeating, each one corresponding to a stage that got skipped.

Overfitting to in-sample data. The backtest is incredible and the live performance is mediocre, because the strategy was fit to noise in the research data. Walk-forward validation catches most of it; honest DSR and PBO computation catches more.

Capacity drag going unnoticed. The strategy worked at $10M, scales to $100M, and its own footprint starts moving prices. Pilot deployment exists for exactly this, and a careful pilot at multiple capital fractions prevents the surprise at full size.

Crowding by other participants. A pattern that was uncrowded in research becomes crowded in production as other quants find the same thing, and IC decays. Retraining cannot recover this one; the only response is a new signal or acceptance.

Regime shift without retraining. Market dynamics change (2008, 2020, 2022) and the strategy was trained on a different regime. It still trades, it just makes decisions on stale patterns. Regular retraining, quarterly at minimum, prevents most of this.

Operational failure. The system crashes, the data feed breaks, order management sends bogus orders. The strategy is fine, but nobody is monitoring, so it fails silently. This is the most preventable killer of the six and it still happens with depressing frequency.

Compliance or risk failure. The strategy trades restricted names, exceeds leverage limits, or violates position concentration rules, and compliance or risk shuts it down. Pre-trade checks prevent it; post-trade monitoring catches what pre-trade checks miss.

Five of the six have a concrete countermeasure: pre-trade risk for operational and compliance failures, walk-forward validation for overfitting, the pilot for capacity, scheduled retraining for regime shift. Only crowding has no permanent fix, and that is precisely why strategies need a lifecycle at all.

Last Thing

Most strategies have a two-to-three-year useful life. Plan for that, and don’t quit your day job the day you go live.

The pipeline above is not aspirational. It is the minimum configuration for any ML trading strategy that touches real money, and skipping any stage lowers your odds of surviving long enough for the strategy to pay for itself — usually via the exact stage you skipped.

The hard part of ML trading is not building the model. It is keeping the system alive through all the stages, all the regimes, all the operational failures, and all the decay. The teams that make it are the ones that planned for the lifecycle on day one.

References

Chan, E. (2013). Algorithmic Trading: Winning Strategies and Their Rationale. Wiley. ISBN 978-1-118-46014-6.

López de Prado, M. (2018). Advances in Financial Machine Learning. Wiley. §11.6 (Combinatorially Symmetric Cross-Validation and the Probability of Backtest Overfitting), §12.4 (Combinatorial Purged Cross-Validation), and §14.7.3 (The Deflated Sharpe Ratio). ISBN: 978-1-119-48208-6. The methodology behind proper walk-forward validation, multiple testing adjustments, and the Deflated Sharpe Ratio is all here.

Bailey, D. H., & López de Prado, M. (2014). “The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality.” Journal of Portfolio Management, 40(5), 94-107. The foundational DSR paper, and essential reading for anyone publishing strategy backtest results.

Harvey, C. R., Liu, Y., & Zhu, H. (2016). ”… and the Cross-Section of Expected Returns.” Review of Financial Studies, 29(1), 5-68. Analyzes factor publication bias and argues for much higher significance thresholds on new factors. Note that it points the opposite way from the 0.5 Sharpe gate above: this paper is about whether a factor should be believed at all, and its bar only goes up. The 0.5 is a practitioner deployment floor, not this paper’s recommendation.