Monitoring and Alerting for Live Trading Strategies

Strategy deployed. Now what? Now you watch it. Most teams do this badly — they check PnL once a day and panic when there’s a drawdown. The right approach is continuous monitoring with alert thresholds tuned to catch decay before it kills you.

There’s a moment, somewhere around the second week of live trading, when the novelty wears off and the operational reality sets in. The alpha is real, or it isn’t. The fills are coming in, mostly. The PnL is positive this week. But there’s a persistent low-level anxiety: is this thing working? Is it working the way I think it’s working? How would I know if it stopped?

That anxiety is what monitoring is for. Not PnL monitoring — that only tells you the answer after the money is made or lost. Structural monitoring: the thing that watches the strategy’s behavior and tells you whether the underlying process is still generating alpha, or whether the alpha is already decaying and the PnL is just the visible tip of a slower iceberg.

Most quant teams treat monitoring as a quarterly review. Some treat it as a daily PnL check. A few treat it as the full-time job it actually is. This article is for the third group: the people who want to catch decay in week 2 instead of week 12, who want to know about fill-rate degradation before the broker calls, who want to see factor half-life compressing in the data before the live Sharpe rolls over.

Four Things Break Independently

A live strategy has four things that can fail on their own, with different failure modes and different responses, so each needs its own monitoring. Here they are, ordered from most obvious to most commonly skipped.

Performance: is the money arriving at the expected rate

The most basic question is whether the strategy is making money at the rate you projected. But you can’t answer it from total PnL, because PnL is path-dependent: a bad month of fills can mask a working strategy, and a great month of fills can mask a dying one. What you track instead are distribution-level quantities — live Sharpe, live information coefficient (IC), live hit rate — each compared to the backtest baseline. They answer whether the underlying signal is still predictive, not whether this month got lucky. If your backtest projected a Sharpe of 1.8 and the live number is 1.5 after three months, that’s within noise. If it’s 0.8, something is structurally wrong.

Implementation-wise, compute these on a rolling window (60 trading days is standard, sometimes 90 for slower strategies) and compare to the backtest distribution. Live 60-day Sharpe below the 25th percentile of the backtest Sharpe distribution: be concerned. Below the 5th percentile: be alarmed.

Risk: where you sit against every limit, right now

Live risk monitoring is not pre-trade risk checking. Pre-trade checks stop new orders from breaking limits; risk monitoring tells you where you currently are on each of them. The quantities to watch in real time cluster around a few exposures: drawdown from the high-water mark, in percent and in dollars; gross leverage (total long plus short over NAV) and net leverage (long minus short over NAV); position concentration — the largest position and the top five as fractions of NAV; sector exposure, with concentration breaches flagged; exposure to the common factors (value, momentum, quality, size, volatility); and rolling beta to the market — the one to watch hardest for long-short and market-neutral books, because when it drifts, your “neutral” is neutral in name only.

A risk monitoring dashboard isn’t a chart. It’s a table: red/yellow/green status on each limit, plus a timestamp for when each status last updated. A green light with no timestamp doesn’t count as green — it might just be three days stale.

Operations: whether the alpha gets a chance to express itself

Your strategy can have perfect alpha and still lose money if the operational layer is broken. These metrics have nothing to do with alpha and everything to do with whether it can cash out. Fill rate — the share of submitted orders that actually fill — is yellow below 85% and red below 70%. Data freshness asks how old the latest market data point is: for end-of-day strategies, older than the previous close is broken; for intraday, older than a minute is worrying. Broker API latency is worth tracking at the median and the p99 of order round-trip time, because latency creep often precedes outright outages. A spike in the order rejection rate means a configuration change or a new account limitation. And reconciliation diffs — how often your internal position state disagrees with the broker’s — follow a simple rule: any non-zero diff is yellow, a persistent diff is red.

Decay: whether the market is figuring it out

This is the dimension most teams skip, and the one that matters most for longevity. Decay monitoring asks: is the underlying factor still predictive, or is the market starting to arbitrage it away?

The core measurement is rolling IC over different windows. Compare current 60-day IC to the historical IC distribution: below the 25th percentile, the factor may be entering a decay phase; below the 10th, it’s probably dying. Note this is the same ruler performance monitoring uses, pointed at a different question — performance monitoring asks “how has this strategy been doing lately,” decay monitoring asks “which way is this factor’s predictive power trending.”

There’s a subtler decay signal: factor half-life. You can estimate it from rolling IC autocorrelation or from the decay rate of rolling Sharpe — an application-specific inference, not a ready-made AFML formula; López de Prado covers the relevant machinery in his treatment of structural breaks in Advances in Financial Machine Learning (Wiley, 2018, ISBN 978-1-119-48208-6), Ch. 17. The short version: every factor has a half-life. Some are months, some are years. Knowing which kind you hold is what lets you tell a 3-month drawdown that is noise from one that is a signal.

From Metrics to Action: Alert Tiers

The four dimensions produce a pile of numbers, and numbers by themselves won’t save you — the mechanism that turns numbers into action will. Alerts are not equal; build tiers.

Yellow (warning) is the “something is off, please look” tier: say, live Sharpe below 70% of expected for two straight weeks. Email plus dashboard, no automatic action. Orange (concern) is the “we should do something” tier: live Sharpe below 50% for a month. Email, Slack, a meeting on the calendar; the strategy keeps running, but new capital is paused. Red (action) is “intervene now”: drawdown past 15%, or Sharpe below 30% for a month. Automatic position reduction, halt of new entries, escalation to the senior risk officer, PagerDuty-grade notification.

The tier names are arbitrary; the discipline is not. Define what triggers each level and what the response is, and make the red-tier response automatic — so that human response time is never the variable that determines your PnL. A useful test: if your monitoring system can’t survive a Friday with nobody checking in, you don’t have monitoring. You have babysitting.

The Decay Dashboard

The most useful single view for an active quant strategy is the decay dashboard — everything that tells you whether the alpha is still alive, on one screen. A rolling 60-day IC chart with horizontal lines at the historical 25th, 50th, and 75th percentiles. A rolling Sharpe chart with the same reference lines. Capacity utilization — how much of theoretical capacity you’re using, since past roughly 80% further growth starts hurting performance (more on this below). Realized slippage per trade against the backtest’s slippage assumption — if actual is twice expected, your net Sharpe is half what you thought. And a drawdown curve with markers at the 5%, 10%, and 20% thresholds.

The dashboard should be live, refreshed at least every 15 minutes during market hours, and viewable from any device including a phone — because the moment you really need it is the moment you’re not at your desk. The underlying queries need to be cheap by design: a dashboard hits them every few seconds during the session, an alert system every minute. The rolling-window math, percentile lookups, tier evaluation, and notification routing belong in the platform layer, implemented once — not copy-pasted into every strategy’s scripts.

The Honest Trap

Drawdowns happen. Drawdowns of more than 20% in a single month happen. The question isn’t whether you’ll have them — it’s whether your monitoring catches them fast enough to act.

It has happened that a strategy lost 25% of NAV in three weeks purely because the team checked PnL once a week and didn’t notice the rolling Sharpe had crossed into the red tier on day 6. By the time they noticed, the recovery path was far longer and harder than it would have been had they pulled the plug on day 7.

The reverse failure mode is just as costly: the team panics at a 6% drawdown, halts the strategy, then watches the alpha reassert the next month while the book sits in cash. So the monitoring system has to distinguish “this is a drawdown, the strategy is fine” from “this is decay, the strategy is dying.” A bare PnL threshold cannot tell the difference. The combination of rolling IC, rolling Sharpe, drawdown, and capacity can.

The discipline: define the alert tiers in advance, while you’re calm and the strategy is making money. Decide what triggers a pause, a halt, an investigation — and write it down. Then when the drawdown comes, you follow the protocol instead of improvising under fire.

The Operations Cadence

Tiers and dashboards cover “when something breaks”; the checklist covers “the rest of the time.” This is the cadence that works in practice — and since it’s a list you genuinely check off item by item, it stays a list.

Daily (every market day, 15 minutes):

• Check fill rate vs. 7-day average
• Check data freshness on every active feed
• Check current drawdown vs. limits
• Skim the top 5 positions and confirm they match the strategy’s intent
• Review any alerts from the previous day

Weekly (Monday morning, an hour):

• Review rolling 60-day Sharpe vs. expected
• Review rolling 60-day IC vs. expected
• Check capacity utilization vs. plan
• Review slippage vs. expected slippage
• Confirm all risk limits are still appropriate (sometimes the strategy drifts and a limit that was 2x safety becomes 1.1x safety)

Monthly (first Monday of the month, half a day):

• Re-estimate factor half-life from rolling IC autocorrelation
• Compare realized factor half-life to expected half-life
• If realized half-life is shortening, discuss whether to tighten the retraining cadence
• Review all positions for stale orders
• Run a mini-validation: take the last month out of the model, replay it, see if live performance matches the out-of-sample backtest

Quarterly (every 3 months, 2 days):

• Re-run the full validation battery on live data
• Check for structural breaks in the factor (López de Prado’s CUSUM test, Ch. 17 of Advances in Financial Machine Learning)
• Recalibrate risk limits based on realized volatility
• Capacity stress test: how does the strategy perform at 1.5x current capital?

This checklist isn’t aspirational; it’s the minimum. Chan’s Quantitative Trading (Wiley, 2009, ISBN 978-0-470-28488-9) discusses money and risk management routines in Ch. 6 that overlap with the weekly and monthly cadence above. His key insight: routines prevent the “I was going to check but something came up” failure mode — the one that quietly kills strategies over 18-month windows.

Capacity: The Slowest Kind of Decay

Capacity keeps appearing in that checklist, and it deserves its own section, because it is the one failure that arrives faster the more successful you are. Capacity is the strategy’s ability to scale without alpha decay. Most strategies have a ceiling: past it, your own trades push prices against you, or you fill at worse prices than the backtest assumed.

The capacity formula in its simplest form — volume daily, turnover annualized — is:

Here $\tau_{annual}$ is annual traded notional as a fraction of AUM; if turnover is already expressed per day, drop the 252. This is a simplified convention that aggregates the portfolio through the “average name.” Plugging in numbers — $50M average daily volume per name, willingness to be 5% of volume, $\tau_{annual}=0.20$ — gives:

That’s the notional you can run before market impact starts hurting. Above it, slippage rises and net Sharpe compresses. So what you monitor is a handful of ratios: current AUM over estimated capacity (utilization); your traded volume over total market volume per name (participation); realized versus expected slippage, segmented by participation rate; and fill rate at different participation rates — if you’re 10% of volume and expect to keep filling at 95%, you probably won’t, which means your orders are too aggressive for the size. When utilization crosses 70%, growth planning starts. When it crosses 90%, growth stops and you evaluate capacity expansion: more names, lower turnover, longer holding periods.

Alerts Have to Reach a Human

An alert that doesn’t reach a human is a wasted alert, so the routing stack is part of the design. Informational events — routine metric updates, completed batch jobs, normal state transitions — go to the dashboard and the log, and bother nobody. Yellow warnings go to email plus dashboard: live Sharpe under 70%, fill rate under 85%, data freshness warnings, with a response expectation of “within the trading day.” Orange concerns add Slack: Sharpe under 50% for a month, drawdown past 10%, reconciliation diffs — respond within an hour. Red actions get PagerDuty-grade paging plus automatic action (position reduction, halt): drawdown past 15%, kill switch triggered, broker API down more than 5 minutes — response within minutes, with the automatic action already taken before anyone has read the page.

The principle is one line: every alert needs a defined recipient, a defined response time, and a defined response action. If you can’t fill in all three for a given alert type, that alert shouldn’t exist. And the failure mode to dodge is alert fatigue: send 50 emails a day and no human reads any of them. Be ruthless about triggers — anything that can be a dashboard update instead of an alert should be one.

When an Alert Fires

Alert fires. Now what? The disciplined response is a fixed sequence. Acknowledge it explicitly — the system should require acknowledgment, or you can’t tell whether anyone saw it. Investigate within the response window: what’s the metric, is it real or a data artifact, persistent or transient? Decide which bucket it lands in: within tolerance (return to normal), yellow (monitor more closely), orange (reduce exposure), red (halt and investigate). Act per the tier protocol, and document both the decision and the action. And if the alert was orange or red, run a post-mortem within 48 hours: what caused it, what we did, what we should change.

The wrong response: the alert fires, you ignore it, and you panic a week later when the drawdown is twice as deep. The right response: the alert fires, you follow the protocol, and you act before the situation gets worse.

Monitoring isn’t a side project. It’s the full-time job that determines whether your strategy survives to year 3. Build the four dimensions, set the tiers in advance, run the checklist on cadence, treat the decay dashboard as your primary view, and wire the alerts into routing that actually reaches people. PnL is the result; monitoring is the practice that keeps the result repeatable.

References

• López de Prado, M. (2018). Advances in Financial Machine Learning. Wiley. Chapter 17 (Structural Breaks). ISBN: 978-1-119-48208-6.
• Chan, E. (2009). Quantitative Trading: How to Build Your Own Algorithmic Trading Business. Wiley. Chapter 6 (Money and Risk Management). ISBN: 978-0-470-28488-9.
• Harris, L. (2003). Trading and Exchanges: Market Microstructure for Practitioners. Oxford University Press. ISBN: 978-0-19-514470-3.