Back Testing

Traders and investors use numerous tools to determine the future director of financial markets.  These tools range from fundamental analysis, where economic, earnings or monetary data which change the current environment, to technical or statistical analysis.  When you are using different tools, many wonder how to gauge the effectiveness of the tools.  One of the best ways to handle this is to back test your strategy.

Back testing is the process of testing a trading strategy on prior periods. Instead of applying a strategy for the time period forward, which could take years, a trader can do a simulation of relevant past data in order to gauge the its effectiveness. Most technical-analysis and statistical analysis strategies are tested with this approach.

For example, you could test a strategy based on the notion that a specific stock only will move a specific percentage away from its 20-day moving average before snapping back to that moving average.

Back testing is a key component of effective trading-system development. It is accomplished by reconstructing, with historical data, trades that would have occurred in the past using rules defined by a given strategy. The result offers statistics that can be used to gauge the effectiveness of the strategy.

Using the information that is gathered, traders can optimize and improve their strategies, find any statistical anomalies, and gain confidence in their strategy before applying it to the real markets. The underlying theory is that any strategy that worked well in the past is likely to work well in the future, and conversely, any strategy that performed poorly in the past is likely to perform poorly in the future.

Back testing can provide plenty of valuable statistical feedback about a given system. Some universal backtesting statistics include:

  • Net Profit or Loss –
  • Time frame – Past dates in which testing occurred.
  • Volatility measures  – Maximum percentage upside and downside.
  • Averages – Percentage average gain and average loss, average bars held.
  • Ratios – Wins-to-losses ratio.
  • Annualized return – Percentage return over a year.
  • Risk-adjusted return – Percentage return as a function of risk.

The first step in back testing is to decide on a combination of price action or technical indicator that you should use to determine if there is a statistical significance.  Traders usually will use current market interaction as a way of finding patterns or anomalies.  There are some software packages that will allow a trader to click on a price bar and the software will describe all the possible scenarios of prior historical patterns. This is a helpful leg up on determining what kind of strategy you would like to test.

Traders should take into account the broad market trends in the time frame in which a given strategy was tested. For example, if a strategy was only back tested from 2009-2010, it may not fare well in a bear market. It is often a good idea to back test over a long time frame that encompasses several different types of market conditions.  Testing a small period is called fitting a curve, and this could create a situation that does not work over a broader period.  Back testing can sometimes lead to something known as over-optimization or curve fitting. This is a condition where performance results are tuned so highly to the past that they are no longer as accurate in the future. It is generally a good idea to implement rules that apply to all stocks, or a select set of targeted stocks, and are not optimized to the extent that the rules are no longer understandable by the creator.

It is important to take into account the specific instrument that is being used in a back test processes.  For example, a stock on a regulated exchange might test differently than an index or a currency.  Instruments that trade 24/7 will usually not have gaps, where instruments that trade on an exchange that gap when they open, might create false signals.

Volatility measures are extremely important to consider in developing a trading system.  If a financial instrument is very volatile, any specific change could wipe you out.  Trading strategies that rely on very large market movements to make money are difficult to employ.

The average number of periods that a strategy holds an instrument is held is also important to watch when developing a trading system. Although most back testing software includes commission costs in the final calculations, that does not mean you should ignore this statistic. Trading strategies that churn numerous trades can fall subject due to slippage and high commissions.

Leverage or gearing to risk is a double-edged sword. The higher the risk and leverage, the more you will likely make or lose.

The average-gain/loss statistic, combined with the wins-to-losses ratio, and profit factor (you average win multiplied by your average win per trade, can be useful for determining optimal position sizing and money management using techniques.

Annualized return is important because it is used as a tool to benchmark a system’s returns against other investment venues. It is important not only to look at the overall annualized return, but also to take into account the increased or decreased risk. This can be done by looking at the risk-adjusted return, which accounts for various risk factors. Before a trading system is adopted, it must outperform all other investment venues at equal or less risk.

Back testing customization is extremely important. Many back testing applications have input for commission amounts, round (or fractional) lot sizes, tick sizes, margin requirements, interest rates, slippage assumptions, position-sizing rules, same-bar exit rules, (trailing) stop settings and much more. To get the most accurate back testing results, it is important to tune these settings to mimic the broker that will be used when the system goes live.

Back testing is not always the most accurate way to gauge the effectiveness of a given trading system. Sometimes strategies that performed well in the past fail to do well in the present. Past performance is not indicative of future results. Be sure to paper trade a system that has been successfully back tested before going live to be sure that the strategy still applies in real time.