Building Cerberus Markets: The Discipline to Distrust Your Own Backtest

There’s a version of this post that leads with a big return number and a clean equity curve. We’re not writing that post. The more honest story — and the one that actually reflects how this project has been run — is about the discipline required to kill your own work, catch your own inflated numbers, and treat “the system did nothing” as a success condition.

We killed the first strategy

Cerberus Markets started as a research platform for paper-trading algorithmic signals across 30 Kraken cryptocurrency pairs. The initial strategy was a classic: a 20/50 moving-average golden cross filtered by RSI divergence and volume confirmation. Straightforward, well-documented in the literature, easy to reason about.

After several months of paper trading and backtesting, we evaluated it honestly and reached an uncomfortable conclusion: it has no real edge. In bear markets it loses to holding cash. In bull markets it badly underperforms buy-and-hold. This is a common failure mode for trend-following signals applied to highly correlated, mean-reverting altcoins — but the evaluation process, not intuition, is what confirmed it.

Rather than tune the parameters hoping for a better result, we declared it dead. Parameter-tuning a losing strategy is how you build a curve-fitted backtest that looks good until it meets real capital.

Building a proper testing rig

Before designing a replacement strategy, we built the infrastructure to evaluate one honestly. That meant pulling roughly two years of daily historical price data for all 30 pairs and implementing walk-forward validation — train on a rolling historical window, test on the unseen subsequent period, repeat across multiple folds. Walk-forward validation is slower and more pessimistic than a simple train/test split; that’s the point. It approximates what actually happens when you deploy a strategy: the market doesn’t care what your in-sample Sharpe was.

The rig makes it difficult to accidentally report inflated results. That matters, because inflated backtest numbers are easy to produce and seductive to believe.

The new strategy: cross-sectional momentum

The replacement approach is structurally different from the golden cross. Instead of evaluating each pair in isolation and asking “is this coin trending?”, cross-sectional momentum asks “which coins are trending most strongly relative to the others?” The distinction matters: you’re always invested in the strongest subset of your universe, not in any coin that clears an absolute threshold.

The specific implementation ranks all 30 pairs by residual momentum — each coin’s return minus its beta-adjusted BTC return, so we’re measuring altcoin-specific strength rather than just BTC exposure in disguise. We hold the top 5 equal-weight, rebalance weekly, and go to cash when nothing in the universe has positive momentum.

The key architectural insight that shaped the whole approach: most altcoins are leveraged Bitcoin proxies. We measured it — the average pair in our watchlist has ~1.3x beta to BTC, and cross-pair correlations spike toward 1.0 during drawdowns. “Five diversified positions” in practice represents roughly two independent bets. Holding five altcoins during a BTC-driven selloff provides almost no diversification benefit.

That measurement motivated a regime gate: when Bitcoin is below its 100-day simple moving average, the entire portfolio goes to cash. No longs, no exceptions. We’re not trying to catch bottoms or rotate into the “best” coin during a broad selloff. Cash is a position.

Catching our own inflated number

Before deploying the new strategy in paper trading, we ran a structured adversarial review of the backtest — explicitly trying to find reasons the results were wrong. This is the step that most backtest write-ups skip.

The review caught something real: our headline return figure was inflated by an implementation artifact in how we’d modeled volatility-based position sizing. The backtest was taking larger positions than the live system would actually be permitted to take. The math was internally consistent; the model just didn’t match the real constraints.

We corrected it. The adjusted, cost-aware, out-of-sample result: approximately +44% net versus ~+25% for buy-and-hold on the six-month out-of-sample window of a two-year walk-forward test (18 months train, 6 months test), with roughly half the maximum drawdown. The Sharpe ratio came in around 0.7. Those numbers are smaller than the original headline — and they’re the ones we trust.

For robustness context: 133 of 144 parameter configurations we tested (varying lookback window, top-K holdings, rebalance frequency, regime gate length) outperformed the buy-and-hold benchmark. The edge is not dependent on a single precisely-tuned parameter set.

A backtest that flatters you is worse than no backtest. We’d rather catch our own inflated numbers than have the market do it for us.

Where it stands now

The cross-sectional momentum strategy is deployed and running in Cerberus Markets’ paper-trading environment. The architecture is purpose-built for portfolio-level decisions: a weekly rebalancer computes target weights, diffs them against current open positions, and emits the resulting orders through the existing paper-trade execution and accounting infrastructure. Position-level stop-losses remain active between weekly rebalances as a floor.

On its first scheduled run, the strategy correctly did nothing. Bitcoin is currently in a downtrend — below its 100-day moving average — so the regime gate triggered and the whole book is in cash. No positions opened, no trades placed. That outcome is exactly what the design specifies, and it’s a reasonable first data point: the gating logic is running and responding to actual market conditions.

No real capital is at risk. The forward paper-trading period is the real out-of-sample test — the one the backtest can’t substitute for. We’ll accumulate paper results over the coming weeks and compare them against the backtest’s distributional expectations before any live trading discussion begins.

The broader lesson

The engineering story here isn’t the return number. It’s the process: measure rigorously, kill work that doesn’t pass, look for ways your own analysis is wrong, and don’t conflate a good backtest with a good strategy. Those habits are harder to build than any particular signal, and they’re what determine whether a system is worth trusting with real capital.

The paper-trading period is ongoing. More updates as the data accumulates.