Benchmark Lab · Methodology

How we measure.

The only useful benchmark is an honest one. This page describes how we collect data, classify market conditions, anonymize strategies, and decide what is safe to publish. It is updated whenever the methodology changes.

Data sources

Benchmark Lab is built from internal benchmark runs stored in our research database. The core fact table uses one row per benchmarked trade with two PnL legs: a naive (paper) result and a realistic, execution-adjusted result that stress-tests slippage, fees, failed or partial exits, and conservative fill assumptions.

Unless separately validated and called out, published “realistic” figures are modeled execution outcomes, not proof of a specific on-chain wallet’s live fills. We do not republish third-party trade feeds.

Anonymization

Strategy configurations are mapped to opaque family_* buckets via a salted, one-way hash. Internal config IDs, names, parameters and entry/exit rules never reach the public surface. Wallet addresses are stripped at the export boundary — not hashed, not redacted, removed.

Buckets with fewer than the minimum sample size threshold are excluded from public output to prevent identification of single configurations.

Market regime classification

Every benchmarked window is classified into one of four regimes along two axes: buyer velocity (high / low) and price volatility (high / low). Classification uses window-level features only — no per-token labels, no sentiment data, no off-chain signals.

Sample sizes per regime are published alongside metrics. We do not report any regime with insufficient coverage.

Metric definitions

See the metrics page for the precise definition of slippage, failed sells, sell timeout, naive-vs-realistic divergence and family performance.

What we do not measure

Benchmark Lab does not generate trading signals, recommend tokens, predict price direction, or estimate future returns. It measures observed execution and observed performance under historical benchmark windows. Past behavior does not imply future behavior.

What we do not publish

  • Strategy configurations, entry rules, or exit rules
  • Internal strategy or config names
  • Per-trade rows, per-wallet data, or wallet addresses
  • RPC endpoints, API keys, or any infrastructure identifiers
  • Currently active or currently running strategy state