David Bailey, Lawrence Berkeley National Lab (retired) and University of California, Davis
Abstract: It is a sad fact that few investment funds, models or strategies actually beat the overall market averages over, say, a 10-year window. Even in academic research work, care must be taken to avoid statistical pitfalls, because: (a) the chances of finding a truly profitable investment design or strategy is very low, due to intense competition; (b) true findings are mostly short-lived, as a result of the non-stationary nature of most financial systems; and (c) it is often difficult to debunk a false claim. Backtest overfitting is a particularly acute problem in finance, both in academic research and commercial development, since it is a simple matter to use a computer program to search thousands, millions or even billions of parameter or weighting variations to find an “optimal” setting. In this talk, we summarize many of these pitfalls, explore why they are so prevalent, and present some tools that can be used to avoid them, including the “False strategy theorem”.