- Discretionary Versus Algorithmic Trading or Investing
- Why This Book?
- Investing Is a Business
- Strategies
- Backtesting—Standard and Walk-Forward Optimization
- Book Organization
This chapter is from the book
This chapter is from the book
This chapter is from the book
Backtesting—Standard and Walk-Forward Optimization
The results in this book are from standard and walk-forward optimizations I performed at the end of the year 2012. The data is the daily closing price of all U.S. common stocks traded from 1990 through 2012. The data includes listed and delisted stocks during that period, to eliminate survivor bias, but is limited to operating companies. The numerous derivative issues such as Exchange Traded Funds (ETFs) and other interest-rate related stocks are excluded. (ETFs are used in the trading section beginning in Chapter 6.) The total number of stocks analyzed is 6,272. The list also includes American Depository Receipts (for stocks traded in foreign stock markets) and other forms of foreign stock replications that are listed in each of the three principal exchanges: the New York Stock Exchange, the American Stock Exchange, and the NASDAQ exchange. Price data is adjusted for splits and capital distributions but not for dividend payments. I use only price in this system because, as technicians have long argued, price includes every aspect of a stock’s value, its history, its emotional appeal, and its place in the world as of each trade.
Standard Optimization
A standard optimization tests results from many variable parameters (such as lookback period in relative strength, rank to buy, and rank to sell) to see which combination gives the “best” answer. The best answer is subject to the preference of the analyst and is usually a combination of profit and potential loss. The final objective, called the objective function, is the statistic by which each test is measured. The highest level of objective function is the best answer. For example, suppose I use net profit as the objective function. As the computer runs through the various parameter combinations, it will arrive at a “best” set of parameters that gives the highest net profit of all the systems. This would then be the best system based on net profit as an objective function. Of course, net profit alone doesn’t account for risk. The MAR ratio then could become the objective function. This is a ratio between compounded annual growth rate percentage (CAGR) and maximum drawdown percentage (MDD). The series of tests for optimal parameters can then select the combination that had the highest MAR ratio. There are many other objective functions that can be used depending on the result that is important to the analyst.
Walk-Forward Optimization
In a walk-forward optimization, the optimization process is similar to the standard method in that it also uses an objective function by which the results of each optimization are measured. It is different, however, in that it optimizes over shorter periods called “runs” within the entire data series and then tests the parameters found to be the best in the short period on a selected portion of the data that is independent and thus “unknown” to the optimization data. It summarizes these “out of sample (OOS)” results to see how well the best combinations from the original data worked in the unknown data. A successful result is a return in the unknown data similar to the return in the known data. In the walk-forward optimization the number of runs is varied to be sure no pattern has been present between time periods, and the percentage of out-of-sample data used in the tests is varied to be sure that the results summarize both long and short periods subsequent to the optimizations. The process of optimizing and testing results in unknown data is thus to see if the original system hypothesis has merit. One that has a high percentage of favorable test results is called “robust.” It has a high chance of continuing to profit in the future.