Walkthrough 03: Deflated Sharpe Ratio — Getting an Honest Performance Estimate

Goal: By the end, you’ll understand why a reported Sharpe ratio from a backtest is almost certainly too high, how the Deflated Sharpe Ratio corrects for this, and how to report honest performance numbers.

Prerequisites: Basic statistics (Sharpe ratio, p-values) and familiarity with the concept of multiple testing (choosing the best from many).

Step 1: What Is This, Really?

You ran 30 strategy variants. The best one has a Sharpe of 1.8. Your investor is excited. You should be worried.

Here’s why: you tested 30 strategies. At the 5% significance level, you’d expect 1.5 of them to show “significant” positive returns just by chance. The best of your 30 might just be the luckiest.

The Sharpe you see — 1.8 — is the maximum of 30 draws from a distribution whose mean is probably near zero. The maximum is a biased estimate of the true maximum. The Deflated Sharpe Ratio (DSR) corrects for this bias.

DSR tells you: “Given that you tested N strategies, what’s the probability your best Sharpe is real, and what’s a deflated estimate?”

Step 2: Why Does This Matter?

Here’s a real scenario. A quant fund publishes a track record: 3.2 Sharpe over 5 years. The marketing looks professional. The backtest documentation is thorough.

Then the strategy starts underperforming. Not dramatically — but consistently. By year 3, the Sharpe is 1.1. By year 5, it’s 0.8.

What happened? The fund tested hundreds of strategy variants over the years. They reported the best. The reported 3.2 Sharpe was the maximum of hundreds of trials. The true expected maximum under the null (no skill) was probably around 1.5.

The DSR of the reported 3.2 might have been 1.2 — still good, but not 3.2.

If you don’t deflate, you’re lying to yourself and your investors about the true expected performance.

Step 3: The Core Mechanism

The DSR formula adjusts the observed Sharpe downward based on:

1. Number of trials (m): The more strategies you test, the more the maximum will be inflated by luck.
2. Sample size (T): With more data, the variance of the Sharpe estimator is lower, so the expected maximum is lower.
3. Return distribution (skewness, kurtosis): Non-normal returns make the Sharpe estimator more volatile, which inflates the expected maximum.

The deflated Sharpe is:

Where $E_{max}[SR | m, T, \gamma]$ is the expected maximum Sharpe under the null hypothesis (true Sharpe = 0), given m trials, T observations, and the observed return distribution’s skewness (γ).

The expected maximum under the null grows with √(2 ln(m) / T) for normal returns, and faster for fat-tailed returns.

Step 4: The Key Formulas

For normally distributed returns: $E_{max}[SR] = \sqrt{\frac{2 \ln(m)}{T}}$

For m=30 strategies, T=756 (3 years of daily data): $E_{max} = \sqrt{\frac{2 \ln(30)}{756}} = \sqrt{\frac{6.9}{756}} = \sqrt{0.0091} = 0.095$

For non-normal returns (accounting for kurtosis γ): $E_{max}[SR] \approx \sqrt{\frac{2 \ln(m)}{T}} \cdot (1 + \gamma^2/4)$

If returns are leptokurtic (γ² > 0), the expected maximum is higher.

Deflated Sharpe: $SR_{D} = \frac{SR_{max} - E_{max}[SR]}{\hat{\sigma}_{SR}}$

For example: observed max Sharpe = 1.8, E_max = 0.095, sample std of Sharpe estimates = 0.3: $SR_D = \frac{1.8 - 0.095}{0.3} = \frac{1.705}{0.3} = 5.68$

Wait — that’s too high. The DSR can exceed the observed Sharpe if the standard error is very small. The meaningful comparison is to other strategies’ DSRs, not to the raw Sharpe.

Step 5: Implementation

import numpy as np
from scipy import stats
from scipy.special import erfinv

def deflated_sharpe_ratio(returns_matrix, risk_free=0.0, periods_per_year=252):
    """
    Compute the Deflated Sharpe Ratio for a set of strategy returns.

    Parameters
    ----------
    returns_matrix : array (n_samples, n_strategies)
        Returns for each strategy
    risk_free : float
        Risk-free rate (annualized)
    periods_per_year : int
        Number of return periods per year

    Returns
    -------
    dsr : array
        Deflated Sharpe Ratio for each strategy
    osr : array
        Observed Sharpe Ratio for each strategy
    """
    n_samples, n_strategies = returns_matrix.shape

    # Observed Sharpe Ratios
    excess = returns_matrix - risk_free / periods_per_year
    mean_excess = np.mean(excess, axis=0)
    std_excess = np.std(excess, axis=0, ddof=1)

    osr = np.zeros(n_strategies)
    valid = std_excess > 0
    osr[valid] = mean_excess[valid] / std_excess[valid] * np.sqrt(periods_per_year)

    # Estimate parameters for the null distribution
    # Under H0 (true Sharpe = 0), the sampling distribution of Sharpe has:
    # mean = 0
    # std = 1 / sqrt(T)
    T = n_samples

    # Compute expected maximum Sharpe under H0
    # For normal returns: E[max] = sqrt(2 * log(m)) / sqrt(T)
    m = n_strategies
    e_max_normal = np.sqrt(2 * np.log(m) / T)

    # Adjust for kurtosis (leptokurtosis inflates E[max])
    # Using the Jarque-Bera-like adjustment
    kurtosis = np.zeros(n_strategies)
    for j in range(n_strategies):
        ret = returns_matrix[:, j]
        if np.std(ret) > 0:
            ret_standardized = (ret - np.mean(ret)) / np.std(ret)
            kurtosis[j] = np.mean(ret_standardized**4)

    # Adjustment factor
    gamma_sq = np.maximum(kurtosis - 3, 0)  # Excess kurtosis
    adjustment = 1 + gamma_sq / 4
    e_max_adjusted = e_max_normal * adjustment

    # Standard error of Sharpe (for the deflated version)
    # Using the sample std of the Sharpe estimator
    se_sharpe = 1 / np.sqrt(T)

    # Deflated Sharpe
    dsr = np.zeros(n_strategies)
    valid = (std_excess > 0) & (se_sharpe > 0)
    dsr[valid] = (osr[valid] - e_max_adjusted[valid] * np.sqrt(periods_per_year)) / (se_sharpe * np.sqrt(periods_per_year))

    return {
        'dsr': dsr,
        'osr': osr,
        'e_max': e_max_adjusted * np.sqrt(periods_per_year),
        'n_strategies': m,
        'n_observations': T
    }

Step 6: How to Read Results

Compare DSRs, not raw Sharpes. If you’re selecting from 30 strategy variants, rank them by DSR, not by raw Sharpe. The DSR accounts for the fact that you selected the maximum.

A DSR > 1.0 means the strategy is likely real. More precisely, a DSR > 1.96 (for T observations) means the strategy is significant at the 5% level after deflating for multiple testing.

Report both raw and deflated. “Our best strategy has an observed Sharpe of 1.8, and a deflated Sharpe of 1.2 after adjusting for 30 trials.” This is honest and informative.

Watch for negative DSR. If your best strategy has a negative DSR, it means even after deflating, the best strategy looks worse than the expected maximum under the null. This is a red flag — either your best strategy is worthless, or you need more data.

Step 7: Common Pitfalls + Where It Fits

Pitfall: Pretending you only tested one strategy. If you tested 100 variants but only report the best one, you MUST deflate. The DSR is non-negotiable in this case.

Pitfall: Using m = 1 (one strategy). DSR with m=1 gives the standard Sharpe. That’s fine — DSR generalizes to the single-strategy case.

Pitfall: Ignoring non-normality. If returns are fat-tailed, E_max is higher than the normal formula suggests. Using the normal formula will understate the inflation and your DSR will be too high. Always adjust for kurtosis.

Pitfall: Comparing DSRs across different m. A strategy with DSR = 1.5 from 10 trials is more impressive than a DSR = 1.5 from 100 trials. The m matters.

Where DSR fits:

• When selecting from many strategy variants
• When reporting track records to investors
• As a correction to the standard Sharpe ratio whenever multiple testing occurred

DSR is part of a broader framework: multiple testing corrections (covered in the Multiple Testing encyclopedia entry) and PBO (covered in Walkthrough 02). Together, they give you an honest picture of your strategy’s true expected performance.

Next Steps

Ready for the next level? → Walkthrough 04: Backtest Overfitting (PBO)

Want the encyclopedia reference card? → Multiple Testing (Encyclopedia)