Start with the question the test answers.
A backtest applies stated rules to historical data. It can reveal whether an idea survives particular assumptions. It cannot establish what an investor would have earned or promise that the pattern will continue. Before looking at a return chart, ask what decision the test is meant to support.
A forecast test and a trading test answer different questions. Predicting direction more often can still lose money if mistakes cost more than correct forecasts earn. A trading result needs a stated position size and exit rule. Otherwise, the connection between a prediction and a portfolio outcome remains unspecified.
Reconstruct what was known at the time.
Consider a hypothetical strategy that trades after an economic release. If the data file contains a later revision, the simulation may give the model information unavailable on the trading date. The same problem appears when a historical company universe excludes firms that later disappeared. A clean file is not automatically a faithful record of a past decision.
Ask for the source timestamp, decision time and first possible execution time. The distinction matters when a rule uses the closing price but assumes an order filled at that same close. Someone reviewing the result should be able to trace a selected trade from its inputs through the execution assumption.
Count the ideas that failed.
A strong result means less when it was selected from many attempts. Bailey and colleagues' paper on backtest overfitting explains how repeated strategy selection can favor noise. Its framework examines the selection process, rather than treating the winning configuration in isolation. An impressive winning test does not tell you how much searching happened before it emerged.
The practical request is a development history. How many variants were tested? Did the researcher change the trading universe after seeing results? Were disappointing runs retained? A record of rejected ideas helps distinguish a reasoned hypothesis from a rule assembled around the best part of the chart.
Keep evaluation separate from development.
Evaluation should respect the order in which information arrives. Hyndman and Athanasopoulos describe time series cross-validation using training observations that precede each test observation. Moving the evaluation point forward gives repeated checks across time. This is useful for forecasting, but forecast accuracy alone still does not validate a trading strategy's economic outcome.
A reserved sample stops being independent once its results guide further rule changes. Recording those changes is more honest than continuing to call the sample untouched. Ask which model version was frozen before evaluation and how that version compares with the one now presented. Holding data aside is useful discipline, not a certificate that selection bias has disappeared.
Stress the execution assumptions.
For a hypothetical short holding period, a small change in spread or delay may matter more than a small improvement in the forecast. Ask when orders can reach the market, what liquidity is assumed and how partial fills are handled. A price touched on a chart does not demonstrate that an order of the modeled size could fill there.
Review commissions and trading costs before calling a result profitable. Fund fees require a separate investor-level analysis. Ask whether the conclusion survives less favorable fills, missed trades and a smaller available position. Each scenario should state what changed. A broad claim that the test is conservative gives the reviewer little to examine.
Read the loss path, not just the endpoint.
Two curves can finish at similar levels while exposing capital to very different conditions. Look at the concentration of gains and the time spent recovering from losses. Ask whether the conclusion depends on one market episode or a few trades that dominate the sample.
The SEC's hedge fund guide explains that leverage and derivatives can magnify gains and losses. A historical chart does not remove that exposure. Nor does a modeled exit guarantee a market will permit the same exit in stressed conditions.
Write down what would invalidate the result.
Before relying on a test, decide what evidence would change the conclusion. Examples might include discovering unavailable inputs, losing the apparent benefit after realistic costs, or failing to reproduce the result from the documented rules. These are review questions, not universal numerical thresholds.
An unresolved assumption belongs in the conclusion. It should not disappear into a footnote once a chart looks persuasive. The useful outcome of a backtest can be a decision to reject an idea, request better evidence or narrow the circumstances in which it deserves further study.
This note presents no actual or hypothetical fund performance. It describes an evaluation framework, not a completed empirical study or a recommendation to invest.
Sources and further reading
Primary sources behind the cited concepts. Practical questions and hypothetical examples in this note are our explanatory interpretation.
Continue the review
Bring the questions to Investor Relations.
Read the investment process overview and due diligence guide, then request materials for eligibility review. Confidential fund evidence remains separate from these public notes.
Request Investor Materials