Signal Vendor Due Diligence: Red Flags Before You Subscribe

Judging a signal vendor is not about whether the numbers look real. It is about which claims you can check yourself. Eight red flags and the evidence each one calls for.

7 of 9 ~8 min

You’ve found a signal vendor promising 45% annual returns with a 3.2 Sharpe. The backtest looks clean. The website has professional charts. They have a 4.8-star rating on Trustpilot.

Asking whether those numbers look real gets you nowhere — a well-polished overfitted backtest looks exactly like a genuine edge. There is only one useful question: which claim can I check myself. Each of the eight red flags below has an evidence request behind it.

Red Flag 1: No Live Track Record

A backtest is not a track record. Anyone can optimize a strategy on historical data and produce beautiful numbers, and the harder they optimize, the better those numbers look. The only meaningful evidence is a live account, independently verified, showing the strategy executing in real time with real money.

What to ask for is specific:

Broker statements showing positions and P&L
Third-party verification through Myfxbook, FX Blue, or similar
Minimum 2-3 years of live results

The 2-3 year floor is not arbitrary: the sample has to span more than one market regime before the curve carries information.

What they typically say: “We’re preparing the audit.” “The live account is new.” “We can’t share that for legal reasons.” All three mean the same thing — no live record that backs up the backtest. The last one deserves extra suspicion, since there is no legal obstacle to showing your own executed trades.

Red Flag 2: “Signals Work on All Pairs” or “All Market Conditions”

A strategy that performs equally well on EUR/USD, GBP/JPY, gold, and Bitcoin across bull markets, bear markets, and high-volatility regimes has only three possible explanations. It is market-neutral, in which case it should have low drawdown and will not be returning 45% a year. It is diversified to the point of being beta, in which case why not just buy an ETF. Or it is overfitted to a market regime that no longer exists.

Real strategies have regimes. A good trend-following strategy suffers in range markets; a good mean-reversion strategy gets stopped out repeatedly in trending ones — not a defect, but the other face of where the edge comes from. So “works in all conditions” translates to “we haven’t measured when it doesn’t work,” which is precisely the part you need. Ask for performance broken out by year, by asset class, and by market regime. If they cannot produce the breakdown, they have never looked at it themselves.

Red Flag 3: No Explanation of Edge Mechanism

“Algorithmic trading” is not an edge. “Quantitative model” is not an edge. “Our proprietary AI” is not an edge. These describe implementation, not the source of the returns.

A real mechanism names the persistent market inefficiency being exploited, specifically enough to be falsified. “Short-put replication strategy that collects premium when implied vol > realized vol” is a mechanism — it also tells you the return is compensation for bearing tail risk, and when it will blow up. “Trend-following on 60+ futures markets, using 3-month lookback, rebalanced monthly” gives you the parameters, so you can build a rough control version yourself. “Carry trade on G10 currency basket, held 5 days, with volatility-adjusted sizing” is another.

What these share is that each could be shown to be wrong. Without that layer you cannot tell whether the edge is real, whether it has already been arbitraged away, or whether it is a statistical mirage.

Red Flag 4: The Numbers Don’t Reconcile

45% annual returns. Sharpe of 3.2. Maximum drawdown of 3%.

At least one of these is lying, and you can tell without looking at the strategy. In liquid markets with reasonable leverage, sustaining returns above 20% a year while holding a Sharpe above 1.5 is a rare achievement in itself. A Sharpe above 3.0 is almost always one of three things: heavy leverage (which amplifies returns and tail risk together, and which the Sharpe ratio is largely blind to, so ask about it separately), backtest overfitting (optimized until it produced these numbers), or cherry-picked periods (a bull market included, the corrections cut out).

The fastest consistency check is return divided by drawdown. 45% against 3% is 15:1. For calibration: among professional managed futures programs, annual return and maximum drawdown tend to sit in the same order of magnitude, and holding that ratio near 1 over a long horizon is already a strong result. 15:1 is off by an order of magnitude. So always require drawdown and Sharpe together — a table reporting only Sharpe usually omits drawdown because that column is embarrassing.

Red Flag 5: No Discussion of Slippage or Transaction Costs

A strategy that trades 50 times a day, backtested with a 0.1 pip spread and no slippage, produces a result with no information in it. At that frequency, costs eating half to nine-tenths of gross profit is routine, which means the cost assumption alone sets the sign of the answer.

Three questions establish the depth: what spread and slippage assumptions went into the backtest; what slippage they expect live; and whether they can show a sensitivity analysis of returns against transaction costs. The third matters most — surviving at 2x assumed costs is genuine robustness, while going to zero at 1.2x means your broker, your execution speed, and your account size all have to match theirs or you get a different outcome. Failing all three makes the backtest incomplete at best and misleading at worst.

Red Flag 6: Vendor Owns the Data Feed

If the signal vendor also sells you the data feed they trade on, you get a conflict of interest and a ready-made front-running channel in the same package. Picture it: you pay $500/month for “low-latency data” that the vendor also uses for their own trading. They can see who subscribes and when. They have better infrastructure and can be filled ahead of you. They control the feed and can let signals fire at convenient moments. None of this requires malice to happen.

Legitimate vendors use independent, well-known sources: Bloomberg, Refinitiv, exchange feeds. If what they are selling you is a proprietary “exclusive data feed,” treat that as something requiring extra explanation rather than as a feature.

Red Flag 7: No Explanation of Risk Management

Signals without risk management are dangerous, and the danger is independent of signal quality. A 60% win rate strategy that sizes positions badly still gets wiped out by some run of consecutive losses — positive expectancy and not going broke are two separate conditions.

Ask what the position sizing methodology is, how they handle consecutive losses, what their maximum drawdown tolerance is, and whether stop-losses are hard rules or discretionary. If the last answer is “discretionary,” what you are buying is one person’s judgment in the moment rather than a systematic signal, and none of the preceding discussion about backtests applies. A vendor who cannot explain their risk management either has no real system or is unwilling to share it. For you the consequence is identical.

Red Flag 8: No Peer Review or Third-Party Audit

The gold standard is an independent third-party audit covering three things: backtest methodology (was purging and embargoing applied? were multiple-testing corrections used?), data (is it clean? were corporate actions handled correctly?), and live results (are they consistent with the backtest claims?).

The first is the easiest to slide past and the most damaging. A vendor can display a live account with complete honesty while that strategy was the best of several hundred variants — the selection alone manufactures the appearance of an edge, and that step is invisible on any broker statement (for how far multiple testing inflates a Sharpe, see Cult of Sharpe). So the question is not only “are these results real” but “how many strategies did you test before keeping this one.” Vendors who deflect scrutiny with “proprietary methodology” or “competitive advantage” are usually hiding this particular layer.

What Verifiable Evidence Looks Like

Turned around, here is what qualifying evidence looks like.

Independent track record: Myfxbook, FX Blue, or a broker statement showing live P&L. The numbers have to match what the vendor claims publicly, and the account has to be old enough — 3+ years — to have discriminating power.

Transparent methodology: a detailed whitepaper or methodology document covering the edge mechanism, backtest assumptions, risk management rules, and failure modes. The test is hard-edged: you should be able to reproduce a roughly equivalent strategy from it. If you cannot, what they wrote is marketing material.

Sensitivity analysis: how returns vary under transaction cost assumptions at 0.5x, 1x, and 2x expected; across time periods including 2008, 2020, and other stress windows; across trending versus range-bound regimes; and under different slippage scenarios. A good one volunteers where the strategy breaks.

Third-party validation: someone with no financial stake has checked the methodology and results — an academic paper, an independent auditor, or a reputable industry publication. A “validation” from the vendor’s own marketing team does not count.

Due Diligence Checklist

Before subscribing to any signal vendor, work through this:

Every missing item is not a failed formality. It is a gap you cannot check, and the vendor is asking you to fill it with faith. In quantitative finance, faith is not a risk management method.

References

Aronson, Evidence-Based Technical Analysis, Ch. 6-8. Wiley, 2006. ISBN: 978-0-471-75671-9.
López de Prado, Advances in Financial Machine Learning, Ch. 11. Wiley, 2018. ISBN: 978-1-119-48208-6.
Markowitz, Portfolio Selection. Wiley, 1959. ISBN: 978-0-631-15999-9.
Investopedia, “Red Flags of Forex Signal Scams,” 2023.