McLean-Pontiff Pre-Registered: The Replication That Publication Created

The title needs a warning label

The phrase “McLean-Pontiff pre-registered” is useful shorthand and technically misleading. McLean and Pontiff (2016) did not conduct a modern registered report in which hypotheses, outcomes, and code were timestamped before data collection. Their contribution is different: they used published predictors as frozen objects and examined what happened in later periods that the original papers could not have used.

That design creates something like a natural cross-validation exercise. The original paper is the discovery stage. The later data are a test of whether the predictor survives after discovery and, eventually, after publication. The design is not perfect, but it is far more informative than asking whether the original in-sample table looked significant.

This distinction matters because “pre-registered” has become a credibility badge. A label cannot substitute for the actual information flow. McLean and Pontiff’s strength comes from freezing the predictors and measuring their later performance, not from a registration document that did not exist.

The 97-predictor panel

McLean and Pontiff assembled 97 variables from published studies of the cross-section of stock returns. For each variable, they followed the original spirit of the portfolio construction and measured performance across three periods: the original in-sample period, a later out-of-sample period before publication, and a post-publication period.

The first period is selected. The second tests whether the predictor worked before the academic result became public. The third tests whether the predictor worked after investors could learn about it. That timing lets the study separate two kinds of decay:

• decay that appears simply because the original sample was a favorable selected sample;
• additional decay that appears once publication informs trading.

The authors report roughly 26% lower returns out of sample and 58% lower returns post-publication relative to the original in-sample benchmark. The difference, about 32 percentage points, is the publication-associated decline in their decomposition. It is not correct to describe the paper as showing a 32% pre-publication decline and a 58% post-publication decline without explaining the denominator and the decomposition.

The numbers are averages across a heterogeneous set of predictors. They are not a universal haircut for every strategy. A high-capacity value factor and a tiny illiquid accrual strategy should not receive the same forecast. The averages are valuable because they establish a prior: original papers tend to overstate future returns.

Why this resembles cross-validation

In machine learning, cross-validation tests a learning procedure on data not used to fit it. In the McLean-Pontiff design, the “learning procedure” is the original academic discovery process. The published paper selects a variable and construction. Later data evaluate that frozen object.

The analogy is not exact. The original authors did not necessarily freeze all choices before looking at returns, so the discovery stage may already be overfit. The later sample can also overlap with the broader literature’s evolving conventions. But the direction is right: later observations are not available to the original paper’s reported estimate.

Let $\mu_I$, $\mu_O$, and $\mu_P$ be mean returns in the in-sample, pre-publication out-of-sample, and post-publication periods. Define

and

The decomposition asks whether the post-publication decline is larger than the decline that would be expected from ordinary out-of-sample disappointment alone. If it is, publication and information diffusion are plausible mechanisms.

The design has an important virtue: publication dates are outside the strategy’s return construction. They define an information event that can be studied without asking the original researcher to rerun the model.

What the test survives from publication bias

Publication bias operates before McLean and Pontiff begin. The 97 predictors are published winners, not a random sample of all tested ideas. A post-publication test therefore cannot tell us how many unpublished anomalies failed. It tells us how published claims perform once selection and time are confronted.

That is still a lot. If the published set were composed of strong, stable predictors, their later returns should remain substantial. Instead, the average effect shrinks. Predictors with stronger original returns tend to show larger post-publication declines, which is exactly what one expects if selection and crowding favor dramatic results.

A literal pre-registration would address a different problem: it would prevent the analyst from changing the test after seeing the confirmation sample. McLean and Pontiff’s design reduces that problem by using an external publication event and an existing published predictor list. It does not erase every possible researcher choice in constructing the later portfolios.

The publication event is economically meaningful

Publication can affect returns through several channels. Investors may read the paper directly. Data vendors may add the characteristic to a product. Other researchers may use the factor as a control or build a variation. Funds may incorporate the signal into a portfolio optimizer. The market does not require every participant to read the original article; information can diffuse through intermediaries.

A simple model is

where $I_t$ is the market’s information set and $q_t$ is aggregate trading demand induced by the signal. Publication expands $I_t$ and tends to increase $q_t$. If $\lambda>0$, the expected return falls as capital crowds the trade.

The effect should be stronger for signals that are cheap to trade and weaker for signals protected by risk, costs, or capacity limits. McLean and Pontiff find more pronounced publication-related declines among predictors connected with high idiosyncratic volatility and low liquidity. Those are precisely the places where the original gross spread is difficult to capture and where new capital can move prices.

The result is not a clean laboratory experiment. Publication dates cluster, market regimes change, and information may leak before formal publication. The study’s contribution is a pattern consistent with the competition story, not a claim that one journal issue caused every basis point of decay.

What the pre-publication period tells us

The pre-publication out-of-sample period is the most underrated part of the design. It measures what happened after the original sample but before the predictor was formally public. A decline here is evidence against taking the original estimate literally even when no publication crowding has occurred.

That decline can come from selection bias, sample-specific luck, changing data quality, or a changing market. It can also come from informal diffusion: working papers circulate before publication, conference presentations expose the idea, and practitioners discover patterns independently.

The pre-publication period should therefore be read as “later evidence before formal publication,” not as a perfectly clean private market. The post-publication period then adds a stronger information treatment, but not an isolated one.

The role of t-statistics

For predictor $j$, define a period-specific return $\bar{f}_{j,s}$ and standard error $SE_{j,s}$. The t-statistic is

A decline from a large in-sample t-statistic to a smaller later t-statistic can reflect lower mean returns, higher volatility, shorter samples, or more conservative standard errors. Comparing t-statistics alone can therefore mislead. McLean and Pontiff also analyze return magnitudes and characteristics of the predictors.

A strategy manager should carry the same discipline into replication. Compare the mean spread, volatility, turnover, drawdown, number of observations, and costs. A t-statistic that falls because volatility doubled says something different from a t-statistic that falls because the mean went to zero.

The original paper’s t-statistic is also selected. Its later counterpart is not independent of the fact that the predictor was chosen for publication, but it is less contaminated by the original sample’s favorable noise. That is why the decline is informative even though it is not a textbook unbiased estimator.

A simple cross-validation ledger

A practical replication ledger can represent each predictor as a frozen model version:

The key word is unchanged. If the later sample requires a new filter, a new sort, or a new benchmark, record the change and treat the result as a new specification. Do not “repair” the old factor and call the repair a replication.

What this teaches about pre-registration

Pre-registration is valuable because it separates confirmation from exploration. McLean and Pontiff teach a complementary lesson: even when pre-registration is absent, an external event can create a useful test if the tested object is frozen and the later sample is protected from redesign.

For current research, combine both ideas. Pre-register the discovery hypothesis and the primary test. Keep a separate exploration log. Freeze the model before opening the confirmation sample. When the result is public, continue monitoring it without changing the rule. The publication date then becomes a live test of post-discovery performance, not merely a citation milestone.

This process also exposes the difference between a factor and a factor recipe. A factor recipe includes the variable, universe, timing, winsorization, sorting, rebalancing, costs, and benchmark. If any of those are changed after the original result, the recipe has changed. A name is not a specification.

The four explanations for decay

McLean and Pontiff’s evidence is compatible with four overlapping mechanisms.

Statistical bias means the original estimate selected favorable noise. A later sample removes part of that noise.

Trading costs mean academic gross portfolios overstate what capital can capture. Costs can rise when more capital trades the same names.

Crowding means publication informs additional investors who trade in the same direction.

Genuine alpha decay means the market’s behavior changes because the underlying information advantage or behavioral mistake becomes less valuable.

The mechanisms are not mutually exclusive. A factor can be real, expensive, crowded, and overestimated all at once. The useful question is not which single label wins. It is how much of the original return survives the complete chain.

What survives the test

A predictor that remains positive after the original sample, before and after publication, has stronger evidence than one that only shines in the discovery period. But “survives” should not mean “still has a nominal t-statistic above 2.” Look for stable direction, monotonicity, plausible exposure, capacity, and net performance.

A factor can have a lower post-publication Sharpe and remain worth holding as a diversifier. Its expected return may be modest, but its covariance with other strategies can improve a portfolio. Conversely, a factor can keep a positive gross mean while becoming too crowded or costly to deploy.

The post-publication test is therefore a ranking tool, not a binary gate. It tells you which claims deserve further work. It does not determine portfolio weights by itself.

A tiny decay calculation

The following code makes the publication gap explicit rather than mixing the 26%, 58%, and 32% figures.

in_sample = 0.0050
pre_publication = 0.0037
post_publication = 0.0021
pre_decay = 1 - pre_publication / in_sample
post_decay = 1 - post_publication / in_sample
publication_gap = post_decay - pre_decay
print(pre_decay, post_decay, publication_gap)

The output is a set of ratios, not a causal estimate by itself. The code is useful because it prevents a common verbal error: calling the 32-point publication gap the entire out-of-sample decline.

The takeaway

McLean and Pontiff provide one of the closest things factor research has to an external exam. The original paper submits a claim. Time supplies new data. Publication supplies new competitors. The average result gets smaller.

Calling the design “pre-registered” is too strong. Calling it ordinary backtesting is too weak. It is a frozen-predictor, post-discovery validation exercise that reveals how much the published literature’s returns survive contact with time and information diffusion.

Use the lesson in your own research: write the rule before opening the holdout, freeze it before publication, and keep monitoring after the market learns. If the result weakens, do not immediately tune it back to life. That is how a replication turns into another discovery search.

The limits of the natural experiment

The publication date is a useful information marker, not a perfect treatment assignment. Working papers circulate before formal publication. Conferences expose results. Practitioners may discover the same signal independently. Some papers take years to move from draft to journal, while others attract attention immediately. The post-publication period therefore measures a change in the information environment, not a clean switch flipped on a single day.

The original predictor list is also selected. McLean and Pontiff cannot observe the anomalies that researchers tried and abandoned, so their study does not estimate the fraction of all ideas that are false. It estimates how published claims perform when later samples and market learning are allowed to speak.

Sample composition changes over time. Market capitalization, exchange structure, accounting standards, index products, short-sale access, and transaction costs all move. A predictor may decay because the underlying institution changed rather than because investors read the paper. That is not a reason to dismiss the result. It is a reason to state the external-validity claim carefully.

The study also inherits measurement choices. Reconstructing a published signal requires decisions about data timing, missing values, delistings, breakpoints, and portfolio weighting. If the later implementation differs from the original in a material way, the result is a replication of a nearby factor recipe, not necessarily the exact original object.

These limitations make the findings weaker as a causal estimate and stronger as a research warning. The average published anomaly does not retain its original magnitude in later data. A manager who assumes that the original magnitude is still available is making an assumption that the evidence does not support.

The right follow-up is a registered, versioned continuation: freeze the original recipe, publish the data and code choices, define the later window before opening it, and report the result even when it is unhelpful. That process converts the natural experiment into a better modern experiment.

What a modern registered continuation would add

A current research team can extend the McLean-Pontiff logic with a genuine registered continuation. Before accessing the confirmation period, freeze the original signal recipe and publish the analysis plan. Declare the primary return measure, the portfolio universe, the cost assumptions, the period split, and the rule for handling missing or revised data.

Register separate hypotheses for persistence and decay. Persistence asks whether the sign and economic relationship remain. Decay asks how the magnitude changes relative to the original estimate. These are not the same test. A factor can preserve its sign while losing most of its return, which is precisely the situation a live portfolio manager needs to understand.

Pre-specify a stopping rule for the monitoring period. Do not retire the factor after one bad month or resurrect it after one good month. Use fixed windows, confidence intervals, and a declared decision rule. If a new implementation is developed, assign it a new version and keep its performance separate from the frozen replication.

The continuation should publish failures and deviations. If the original data are no longer available, state which fields were reconstructed and how the change might affect the result. If the signal cannot be implemented after borrow and capacity costs, report that as an economic replication failure rather than hiding it behind a gross return.

This is how a historical paper becomes a research asset: not by preserving the original headline, but by preserving the information flow that lets later researchers learn from both survival and decay.

References

• McLean, R. D. and Pontiff, J. (2016). “Does Academic Research Destroy Stock Return Predictability?” The Journal of Finance, 71(1), 5-32.
• Harvey, C. R., Liu, Y., and Zhu, H. (2016). “…and the Cross-Section of Expected Returns.” The Review of Financial Studies, 29(1), 5-68.
• Aronson, D. (2006). Evidence-Based Technical Analysis. Wiley, Chapters 9-12.
• López de Prado, M. (2018). Advances in Financial Machine Learning. Wiley, Chapters 7, 11, and 12.
• Chen, A. Y. and Zimmermann, T. (2021). “Open Source Cross-Sectional Asset Pricing.” Critical Finance Review, 10(1), 1-62.