Backtesting
Test predictions with information available at the time
Backtesting compares forecasts or prespecified rules with outcomes that were unavailable when each forecast or decision would have been made. Risk-model backtests examine coverage, dependence and tail severity; strategy backtests additionally require implementable holdings, costs and execution assumptions. Repeatedly selecting the best rule on the same historical period turns apparent evaluation into training.
Choose this for evaluating historical predictive or decision performance when point-in-time inputs and later outcomes can be reconstructed. Reserve final evaluation data and match the test to the intended use and horizon.
Strengths
- Tests operational forecasts against realised outcomes
- Reveals temporal instability and benchmark-relative performance
Limitations
- Look-ahead, survivorship and selection bias can inflate results
- Rare tail failures and changing regimes limit statistical power
Know the boundary
A successful historical backtest does not guarantee future performance. Correct VaR coverage alone does not establish independence or tail-loss accuracy.