Regime Detection: Finding the Market States Your Factor Works In

A factor that works beautifully in trending markets often fails in range-bound markets. A factor that works in high-volatility environments often underperforms in low-volatility regimes. Pretending markets are stationary when they’re not is one of the most expensive mistakes in quant investing.

Regime detection is the practice of identifying market states and understanding factor performance conditional on those states.

Why Market Regimes Matter for Quant Strategies

Factor performance is regime-dependent. Momentum works in trending markets, fails in mean-reverting markets. Value works in certain macro regimes and fails in others. If you don’t know which regime you’re in, you don’t know whether your factor will work.

Risk management requires regime awareness. A strategy that’s appropriately risky in a low-vol regime can be dangerously over-leveraged in a high-vol regime. Position sizing should adjust for regime.

Tail events cluster in regimes. Financial crises don’t happen in isolation — they occur in specific regimes (high correlation, low liquidity, forced selling) that are identifiable in advance.

The Types of Regimes

Volatility regimes: High-vol vs. low-vol regimes are the most common. VIX > 25 is typically high-vol; VIX < 15 is low-vol. Factors that work in low-vol may not work in high-vol.

Trend vs. mean-reversion regimes: Trending markets (past returns predict future returns) favor momentum factors. Range-bound markets (returns revert to mean) favor mean-reversion factors. You can identify this with autocorrelation metrics.

Correlation regimes: In normal markets, assets have moderate pairwise correlations. In crisis regimes, correlations spike toward 1 (everything sells off together). This is critical for diversification.

Liquidity regimes: Normal markets have liquid markets with tight bid-ask spreads. Crisis regimes have wide spreads and thin order books. High-frequency factors are especially sensitive to liquidity regimes.

Methods for Regime Detection

Method 1: Volatility-Based Regimes (Simplest)

Use rolling volatility to classify regimes:

import numpy as np
import pandas as pd

def classify_volatility_regime(returns, window=60, low_thresh=0.01, high_thresh=0.025):
    """
    Classify volatility regimes based on rolling vol.

    Returns 'low', 'normal', or 'high' vol regime.
    """
    rolling_vol = returns.rolling(window).std() * np.sqrt(252)

    regimes = pd.Series(index=returns.index, data='normal')
    regimes[rolling_vol < low_thresh] = 'low'
    regimes[rolling_vol > high_thresh] = 'high'

    return regimes


def classify_trend_regime(prices, window=60):
    """
    Classify trend vs. mean-reversion regimes.

    Use the fraction of price above/below moving average.
    High fraction = trending; Low fraction = range-bound.
    """
    ma = prices.rolling(window).mean()
    price_ma_ratio = prices / ma - 1

    regimes = pd.Series(index=prices.index, data='range')
    regimes[price_ma_ratio > 0.02] = 'trending'
    regimes[price_ma_ratio < -0.02] = 'trending_down'

    return regimes

Method 2: Hidden Markov Models (More Sophisticated)

HMMs infer a finite number of hidden states from observable data. For market regimes, the hidden states are the regimes and the observables are returns, volatility, and other market features.

from hmmlearn import hmm
import numpy as np

def fit_market_hmm(returns, n_states=3, features=None):
    """
    Fit a Hidden Markov Model to identify market regimes.

    Parameters
    ----------
    returns : Series or array
        Asset returns
    n_states : int
        Number of regime states to identify
    features : DataFrame, optional
        Additional features (volatility, correlation, etc.)
    """
    if features is not None:
        X = np.column_stack([returns.values, features.values])
    else:
        X = returns.values.reshape(-1, 1)

    X = np.nan_to_num(X, nan=0.0)

    model = hmm.GaussianHMM(n_components=n_states, covariance_type='full',
                              n_iter=1000, random_state=42)
    model.fit(X)

    hidden_states = model.predict(X)
    state_probs = model.predict_proba(X)

    return {
        'model': model,
        'hidden_states': hidden_states,
        'state_probs': state_probs,
        'means': model.means_,
        'covars': model.covars_
    }


def analyze_regime_factor_performance(factor, returns, regimes):
    """
    Analyze factor IC conditional on each regime.
    """
    results = {}
    for regime in regimes.unique():
        mask = regimes == regime
        if mask.sum() < 30:
            continue

        ic = np.corrcoef(factor[mask].values, returns[mask].values)[0, 1]
        results[regime] = {
            'ic': ic,
            'n_samples': mask.sum(),
            'mean_return': returns[mask].mean(),
            'vol': returns[mask].std()
        }

    return pd.DataFrame(results).T

Regime Detection for Factor Allocation

Once you’ve detected regimes, you can do factor rotation: allocate more to factors that work well in the current regime and less to factors that don’t.

For example, if you’re in a high-vol regime:

• Reduce momentum factor exposure (momentum crashes in high-vol)
• Increase defensive factor exposure (low-vol, quality)
• Reduce overall leverage

If you’re in a trending regime:

• Increase momentum factor exposure
• Decrease mean-reversion exposure

Common Pitfalls

Using too many regimes. A 5-regime HMM might fit the data well but be impossible to trade — by the time you identify a regime, it’s already changed. 2-3 regimes is usually sufficient.

Regime prediction lag. Most regime detection methods are backward-looking. By the time you identify a regime change, the market may have already moved. Use forward-looking indicators (macro data, volatility term structure) when available.

Assuming regime stability within the detected regime. Even within a “high-vol” regime, conditions vary. Don’t over-leverage just because you’re in “high-vol” — you’re still in a distribution.

Where It Fits

Regime detection informs risk management (how much to leverage given current regime), factor allocation (which factors to tilt toward), and cross-sectional stability (whether factor IC is stable across regimes or only works in specific ones).

References

• Hamilton, Time Series Analysis, Ch. 22-23. Princeton University Press, 1994. ISBN: 978-0-691-04289-3.
• Ang, Asset Management, Ch. 11. Oxford University Press, 2014. ISBN: 978-0-19-995932-3.
• Kritzman, “Regime Shifts: Implications for Dynamic Strategies,” Financial Analysts Journal, 2012.