Journal of Computational Finance

Risk.net

The probability of backtest overfitting

David H. Bailey, Jonathan M. Borwein, Marcos López de Prado and Qiji Jim Zhu

  • A general framework for modeling the IS and OOS performance using probability is proposed.
  • Results indicate that CSCV provides reasonable estimates of PBO, with relatively small errors.
  • CSCV provides both a new and powerful tool in the arsenal of an investment and financial researcher.

ABSTRACT

Many investment firms and portfolio managers rely on backtests (ie, simulations of performance based on historical market data) to select investment strategies and allocate capital. Standard statistical techniques designed to prevent regression overfitting, such as hold-out, tend to be unreliable and inaccurate in the context of investment backtests. We propose a general framework to assess the probability of backtest overfitting (PBO). We illustrate this framework with specific generic, model-free and nonparametric implementations in the context of investment simulations; we call these implementations combinatorially symmetric cross-validation (CSCV). We show that CSCV produces reasonable estimates of PBO for several useful examples.

To continue reading...

You need to sign in to use this feature. If you don’t have a Risk.net account, please register for a trial.

Sign in
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an indvidual account here: