OBSERVATION
Conflict trades looked terrible.
Long trades taken against the broader trend context showed much larger adverse excursion and very weak outcomes.
Anonymized internal research
The Evidence Library.
BackTest Forensics is built from a research process that has been used to interrogate real futures-strategy behavior. These cases show what the process has uncovered, what explanations failed, and how each finding was converted into the next experiment.
These are research examples, not performance promises. Exploratory, post-hoc and counterfactual findings are labeled as such. The point is the depth and discipline of the investigation.
Case 01 · Hidden liquidity failure state
The backtest showed losses. Order-book evidence exposed a condition the strategy never measured.
Original evidenceA concentrated losing window in an intraday futures strategy.
Added evidenceMBO-derived liquidity-authenticity, cancellation and persistence behavior.
FindingA low-authenticity, cancel-heavy state contained no winners in the exploratory flagged cohort and captured the large majority of the loss.
Next experimentTest whether exposure to that state can be reduced without eliminating the strategy’s profitable population.
Research status: exploratory. The finding justified validation; it did not justify declaring a universal spoof filter or a live trading rule.
Case 02 · Entry problem or exit problem?
MAE/MFE changed the diagnosis.
“The long trades are losing” is a description. EdgeQuery asked whether those trades ever generated enough favorable movement to blame the exits.
Long losersAbout 107 points mean MAE versus about 54 points mean MFE.
Short losersSimilar adverse excursion, but materially more favorable movement than the losing longs.
MeaningThe full-sample evidence pointed more strongly toward directional path/entry quality than a strategy-wide exit defect.
ContradictionA later portion showed more favorable excursion, keeping profit protection and exit handling alive as a secondary hypothesis.
The important result was not a stop value. It was a narrower explanation of what was actually wrong—and what still remained unresolved.
Case 03 · A tempting explanation failed
Trend conflict amplified the damage. It did not fully cause it.
CONTROL
Aligned longs were tested separately.
If conflict were the cause, alignment should have repaired the long side.
RESULT
The easy story did not survive.
Aligned longs also remained materially unprofitable. EdgeQuery rejected “just avoid conflict” as an incomplete explanation.
This is one of the most important things a research engine can do: prevent a plausible historical correlation from becoming an unjustified strategy rule.
Case 04 · Hidden regime interaction
The strongest state was not the state that looked most “perfect.”
Inside an otherwise favorable trend/GEX environment, an exploratory propagation grouping produced about +$66.80 per trade with a 4.55 profit factor. The later portion of that group remained around +$60 per trade with a 3.72 PF. Fully synchronized/mature states were negative in both subperiods.
Was the edge tied to transition and propagation rather than static alignment?
Did the grouping survive after the largest winner was removed?
Was the effect stable across early and later portions?
Could the pattern be replicated prospectively?
Research status: post-hoc forward hypothesis. Because the grouping was selected after inspection, EdgeQuery preserved it as a candidate mechanism rather than calling it validated alpha.
Case 05 · Counterfactual risk experiment
A safer experiment emerged without pretending the strategy had been “fixed.”
After long-side weakness survived a chronological challenge, EdgeQuery tested a mechanical reduction in long exposure. A historical 50% long-size counterfactual moved total sample P&L from about −984 to +160 and reduced historical max drawdown from about 1,606 to 930.
WHAT IT SHOWED
Risk concentration mattered.
The long side was contributing disproportionately to damage in the historical sample.
WHAT IT DID NOT SHOW
It was not a forecast.
The counterfactual assumed linear scaling and unchanged fills/behavior. It was explicitly treated as in-sample.
NEXT STEP
Forward-test with a shadow control.
Retain the unchanged strategy as a control while testing lower long exposure prospectively.
Evidence states
A finding is not “true” merely because it is interesting.
BackTest Forensics should tell you not only what it found, but how much trust the current evidence deserves.
OBSERVED
Descriptive evidence.
A measurable pattern exists in the submitted sample.
EXPLORATORY
Candidate explanation.
The relationship is interesting enough to test but has not earned promotion into a rule.
CHRONOLOGICALLY SURVIVED
Persisted across time split.
The finding did not disappear when challenged with a later portion of the historical data.
OUT-OF-SAMPLE
Independent validation.
The hypothesis survives data that did not participate in discovery.
LIVE / PAPER CONFIRMED
Behavior persists in production-like observation.
The mechanism continues to appear under paper or live conditions.
REJECTED
An attractive explanation failed.
The evidence chain records the failure so the same hypothesis does not keep being rediscovered.
The research doctrine
Discover. Challenge. Preserve what survives.
The goal is not to create the most impressive historical story. It is to identify mechanisms that remain useful after we try to break them.
Do not optimize the symptom first.
Understand the failure path before changing the rule.
Do not destroy the winners.
Every proposed loss filter should be challenged against the profitable population.
Do not hide uncertainty.
Post-hoc, exploratory and counterfactual findings remain labeled until stronger evidence exists.