How A Year Of Backtesting Killed Our Best-Looking Indicators

One year of historical data ran against every signal we use. The pretty ones lost. The boring ones won. Here's what survives backtest scrutiny — and why most indicators on Twitter don't.

The take-profit that never fired

A long signal we'd been running for months had a 3% take-profit hardcoded. The signal itself looked sharp: 61% accuracy at 4h horizon, decent sample size, clean entry pattern. By every looking-at-it metric, it was a winner.

Production results said otherwise. Twelve trades, nine stop-loss hits, three breakeven exits. Total: negative.

The backtest revealed the gap immediately. Average maximum favorable excursion across all twelve trades: 0.61%. The 3% target was unreachable. The price never extended that far before reversing. We'd been waiting for a payoff that doesn't exist in the data.

MFE and MAE — the metrics nobody cites

Most strategy posts talk about win rate, profit factor, Sharpe. They don't talk about maximum favorable excursion (how far in your favor the trade went at peak) or maximum adverse excursion (how far against you it went at worst).

Those two numbers tell you whether your stop-loss and take-profit are realistic for the strategy.

• If your TP is > average MFE: you'll never hit it. Trades expire or get trailed early.
• If your SL is < average MAE: noise stops you out. The strategy didn't fail — the SL was too tight.
• Average MFE < Average MAE: the strategy itself has negative expectation. Walk away.

What the year of backtest revealed

We ran 365 days of historical signal data — institutional flow, on-chain, derivatives positioning — against every combo we track. 63 combos in total. The accuracy histogram surprised us:

• 8 combos at >75% accuracy at 4h: mostly DOWN-direction (shorts), driven by liquidity exhaustion + funding squeeze patterns.
• 15 combos at 60-75%: tradable with discipline.
• 40 combos at 50-60%: coinflips with extra steps. Many were our "best" indicators by gut feel.

The split surprised us. Most shorts-side combos beat most longs-side combos at high accuracy. That's not a market opinion — it's what the year of data said.

The optimal TP/SL the data computes

For each combo, we computed optimal TP and SL using the MFE/MAE distribution. The result rarely matched our intuition:

• A combo with average MFE 0.30% wants TP around 0.30%. Not 1%. Not 3%.
• A combo with average MAE 0.15% wants SL around 0.40-0.60% (give noise some room). Not 0.30% (touched too often). Not 1% (lets losers run).
• R:R isn't a starting parameter. It's a derived consequence of MFE/MAE distribution.

What we kill, what we keep

The decision rule we apply after backtest:

• Accuracy at intended timeframe < 55% → kill (or watch-only).
• Accuracy > 55% but average MFE < transaction costs → kill (the edge gets eaten).
• Accuracy > 60% AND MFE > 2× MAE → keep.
• Anything in between → require a confirmatory regime gate (bull / bear context filter) before promoting to live.

Out of 63 combos backtested, 23 made it to production. The rest are either disabled or held in observation mode — logging would-be trades without executing them, to keep validating against future data.

Why your favorite indicator probably loses money

Indicators that look great on a chart usually have one of these problems:

• Look-ahead bias: the lines are smooth because they incorporate future data.
• Survivorship: you remember the times it worked and forget the times it didn't.
• Curve-fit: the parameters were tuned to the chart you're looking at and won't generalize.
• Wrong horizon: it predicts the next 1h move correctly but you trade it on 5m and the noise eats the edge.

None of these survive a year of backtest with proper out-of-sample validation. Most indicators in your watch-list won't either.

If a signal looks too clean, the test is: run it across a full year of data with realistic fees and slippage. If it still works, you found edge. Usually it doesn't, and you saved yourself a quarter of losses.