Back-testing and System Performance
In order to trust a trading or investment system, you need to gain confidence that it has performed well in the past. This article is designed to give you some pointers to help you understand whether the system you are planning to implement will perform according to your expectations.
To conduct in depth research using technical analysis techniques is actually quite a complex task. If you wish to trade longer hold periods, say, greater than a year, you may also wish to use fundamental analysis techniques.
To effectively conduct technical analysis research you will need the following:
Analysis is conducted on price action charts at a particular point in time to determine whether right now is a high probability time to open a position in the price action that is being analysed. The price action can be that of a stock, commodity, currency, index, bond etc. The price action on a chart will show historical data reflecting a certain period of time per bar or candle ranging from 1 minute to 1 month. Analysis can involve price patterns, momentum indicators, volume indicators plus many more. The objective of analysts is to find trends that have just started or to find price action that has reached a level of support or resistance. Their motivation is to find a profitable trade or to avoid a loss trade, the latter being the thinking of a trader that will typically lose money over the long haul. Analysts without a method that incorporates objective entry and exit rules will spend hours and hours subjectively trying to find trades that will avoid a loss. Research, on the other hand, is a structured process that a market researcher will conduct with the stated aim of determining objective entry and exit rules based on historical data that will provide the trader with an edge in the market that will put the odds in the trader’s favour when engaging the market in the future. The trader of a researched trading system does not try to find profitable trades or try to avoid loss trades. This trader’s focus is not on analysis; it is on following regular processes that result in engaging the market according to the rules of the researched trading system. This trader knows that analysis is dangerous as it leads to inconsistency. Successful discretionary traders are also the product of research. Their research is a more subtle process that has been completed over many years of experience in the markets. Successful discretionary traders follow intuitive rules on a consistent basis. Measuring your Trading System
The objective of doing this research is to define a set of unambiguous entry and exit signals that:
Rather than risking ‘real’ money in the markets, a lot of traders begin by paper trading. They write down every detail of their trade in a trading diary, including their position sizing, stop loss, and profit taking procedures. They imagine that they’re really going to take the trade and the record their entry price and monitor the share as time goes on. This has the effect of letting the trader gain some valuable experience without the risk that goes along with investing. There are a couple of problems with this idea though. In ‘The Tao of Trading’, by Robert Koppel, he says: "You can't kid yourself in trading. You have to deal with who you really are, and take responsibility for all your shortcomings, which the markets have a way of revealing rather starkly. You have to confront all your fears and tame them. You have to check your ego at the door. You learn from each experience. What I have learned is this: Patience and diligence are rewarded. Profits will eventually accrue if you do the right thing and stick with it." Paper trading sometimes allows people to kid themselves that they are actually more skilled than reality would suggest. It is wise to try at least a few trades on paper at first, especially if you are testing a new trading system. This is especially true if you are beginning to trade a new instrument that you haven’t traded before, such as options and short selling. This will give you a chance to understand how these instruments work before risking your capital. However, no matter how clever we are at paper trading, there is nothing like putting your own money into the market and experiencing the ultimate emotional roller coaster. Our emotions have a very real impact on our ability as traders. Gaining Confidence in Your System
“One of the most important things in trading the markets is finding that critical balance between a good level of confidence in your trading and a real sense of perpetual humility." (‘The Best: Conversations with Top Traders’ by Marder and Dupee) Some people find by trading small packages, or even just buying or selling one or two shares at a time can help them develop confidence in their system. CFD’s (Contracts For Difference) can help you to do this without paying for brokerage. CFD’s allow you to deal on share prices without having to physically settle on the trade. With CFD’s you are trading a contract that represents the share. This has many benefits. One of the main tangible benefits is that there is often no brokerage charged for these types of transactions. Other fees may be charged such as a holding fee for keeping the trade open overnight, however, this cost is usually quite minimal in comparison to the full service fees charged by some brokers. The benefit of not paying brokerage is of course, immediately apparent. One of the side benefits is that you can test out your theories in real-life, without being overly penalised for gaining this knowledge. This is one way to gain trading experience when using a new system or just regaining some confidence. CFD’s have literally caused a trading revolution. Traders have flocked to this new tool to gain access to leverage, and develop the ability to go long and short on a variety of tools such as the ASX Top 200 shares. Other possibilities include CFD’s on stock market indices, foreign exchange and metals. Another way to gain confidence is to enter the annual ASX trading competition. Information about this competition is available through www.asx.com.au. This will help you to be answerable for your trading decisions because you must register your trades with a third party. This has worked well for many traders. Portfolio Research
In order to calculate the amount of risk you are prepared to take and the precise position sizes for your researched trading system, you need to merge combinations of open trades into portfolios. This portfolio is constructed according to the trading system, risk management and money management rules as devised above. During this phase you will actually be researching your maximum drawdown thresholds according to your money management rules. This is where the combination of the trading system expectancy, the risk management rules and the money management rules all come together. Altering your money management rules during this portfolio research phase will lead to different levels of maximum drawdown. Which level you decide will depend on your risk appetite. This will depend on how much drawdown, that is, how much money being given back to the market, you can handle? Did I mention that drawdown cannot be avoided? So don’t try to achieve drawdown of 0%! You might define different maximum drawdown levels which are determined by changing your position sizes. These levels might range from 5% to 50% depending on the markets and terms being traded. Each portfolio should be viewed diagrammatically over time by generating equity curves for each portfolio noticing when the drawdown occurs. These equity curves can be compared to the overall market to measure how well the portfolios have performed in different types of market conditions. When this phase is complete you will be ready to take your methodology to the market. If you would like to follow a trading plan template that asks you specific questions to help you consider all of the issues that are important, you can find one in my book Trading Secrets. Read more articles by Louise Bedford. Read more articles by Stock Investor Journal. Get data to assist you with your trading.
- Access to quality historical price and volume data. A minimum of 5 years is necessary. The longer the better.
- A minimum of two years of technical analysis experience to gain knowledge of different types of indicators and tools.
- Some trading experience, preferably with a researched system.
- Computer programming skills.
- Good working knowledge of a spreadsheet application.
- Basic statistics knowledge.
- Time (lots of it). If you are starting from scratch, typically around two full-time person years is required to complete a profitable and workable methodology.
- Access to mentors while going through your research.
- Establish your term for trading: intraday, short term: 1 - 5 days, short term: 1 - 3 weeks, medium term: 2 -10 weeks, medium term: 4 - 40 weeks, long term: 9 - 18 months, long term: > 18 months.
- Direction of trade: long or short or both.
- Market to trade: equities, equity derivatives, index futures or derivatives, foreign exchange, commodity futures or bonds. Essentially you need to decide whether you wish to trade leveraged or non-leveraged markets. When you are starting, it is best to trade non-leveraged areas and slower moving markets, until you are ready to take off your training wheels.
- Instruments to trade: all equities, only volatile equities, large cap equities, equity ETOs, equity warrants, instalment warrants, selection of commodity futures, etc.
- Exchanges to trade: local or overseas, multiple exchanges.
- Average move that you wish to capture. The more leveraged the market that you trade the smaller the market move that you need to capture and the shorter the hold period need be. Longer term systems need to capture larger market moves.
- Minimum winning rate. The larger the market move trying to be captured the lower the winning rate need be.
- Minimum Profit Ratio, that is, the minimum ratio of profit trade size to loss trade size. The longer the term you wish to hold trades the higher this will need to be.
- Frequency of trading. The more frequently you trade the more time you will require for your whole trading business and the smaller the market move that you will need to capture.
Analysis is conducted on price action charts at a particular point in time to determine whether right now is a high probability time to open a position in the price action that is being analysed. The price action can be that of a stock, commodity, currency, index, bond etc. The price action on a chart will show historical data reflecting a certain period of time per bar or candle ranging from 1 minute to 1 month. Analysis can involve price patterns, momentum indicators, volume indicators plus many more. The objective of analysts is to find trends that have just started or to find price action that has reached a level of support or resistance. Their motivation is to find a profitable trade or to avoid a loss trade, the latter being the thinking of a trader that will typically lose money over the long haul. Analysts without a method that incorporates objective entry and exit rules will spend hours and hours subjectively trying to find trades that will avoid a loss. Research, on the other hand, is a structured process that a market researcher will conduct with the stated aim of determining objective entry and exit rules based on historical data that will provide the trader with an edge in the market that will put the odds in the trader’s favour when engaging the market in the future. The trader of a researched trading system does not try to find profitable trades or try to avoid loss trades. This trader’s focus is not on analysis; it is on following regular processes that result in engaging the market according to the rules of the researched trading system. This trader knows that analysis is dangerous as it leads to inconsistency. Successful discretionary traders are also the product of research. Their research is a more subtle process that has been completed over many years of experience in the markets. Successful discretionary traders follow intuitive rules on a consistent basis. Measuring your Trading System
The objective of doing this research is to define a set of unambiguous entry and exit signals that:
- Have a positive expectancy.
- Are repeatable in the future.
- Are practicable for you to engage the market.
- The percentage winning rate, i.e. how often the system produces profit trades. This measurement is meaningless on its own. It is used in combination with the next metric.
- The relative size of profit trades to loss trades, known as the Profit to Loss ratio. For example, a Profit to Loss Ratio of 2.5:1 means that profit trades are 2.5 times larger than loss trades.
- The average percentage move per trade, inclusive of both profit and loss trades.
- The number of trades per period. This measures opportunity to trade. Short-term systems should trade between 2 and 5 trades per month per stock / market. Medium-term systems may only provide 2 - 5 trades per annum per stock.
- Number of consecutive loss trades. This provides an insight into how difficult the system may be to trade from a psychological viewpoint. It also prepares the trader for a string of losses, which will be easier to accept when it occurs.
- Number of consecutive profit trades. This provides an insight into how quickly a portfolio can rise out of drawdown.
- Statistical metrics such as standard deviation, arithmetic mean, median, skew, trimmean, kurtosis and others. These can provide an insight into how volatile the trading system results can be for individually researched entities.
Rather than risking ‘real’ money in the markets, a lot of traders begin by paper trading. They write down every detail of their trade in a trading diary, including their position sizing, stop loss, and profit taking procedures. They imagine that they’re really going to take the trade and the record their entry price and monitor the share as time goes on. This has the effect of letting the trader gain some valuable experience without the risk that goes along with investing. There are a couple of problems with this idea though. In ‘The Tao of Trading’, by Robert Koppel, he says: "You can't kid yourself in trading. You have to deal with who you really are, and take responsibility for all your shortcomings, which the markets have a way of revealing rather starkly. You have to confront all your fears and tame them. You have to check your ego at the door. You learn from each experience. What I have learned is this: Patience and diligence are rewarded. Profits will eventually accrue if you do the right thing and stick with it." Paper trading sometimes allows people to kid themselves that they are actually more skilled than reality would suggest. It is wise to try at least a few trades on paper at first, especially if you are testing a new trading system. This is especially true if you are beginning to trade a new instrument that you haven’t traded before, such as options and short selling. This will give you a chance to understand how these instruments work before risking your capital. However, no matter how clever we are at paper trading, there is nothing like putting your own money into the market and experiencing the ultimate emotional roller coaster. Our emotions have a very real impact on our ability as traders. Gaining Confidence in Your System
“One of the most important things in trading the markets is finding that critical balance between a good level of confidence in your trading and a real sense of perpetual humility." (‘The Best: Conversations with Top Traders’ by Marder and Dupee) Some people find by trading small packages, or even just buying or selling one or two shares at a time can help them develop confidence in their system. CFD’s (Contracts For Difference) can help you to do this without paying for brokerage. CFD’s allow you to deal on share prices without having to physically settle on the trade. With CFD’s you are trading a contract that represents the share. This has many benefits. One of the main tangible benefits is that there is often no brokerage charged for these types of transactions. Other fees may be charged such as a holding fee for keeping the trade open overnight, however, this cost is usually quite minimal in comparison to the full service fees charged by some brokers. The benefit of not paying brokerage is of course, immediately apparent. One of the side benefits is that you can test out your theories in real-life, without being overly penalised for gaining this knowledge. This is one way to gain trading experience when using a new system or just regaining some confidence. CFD’s have literally caused a trading revolution. Traders have flocked to this new tool to gain access to leverage, and develop the ability to go long and short on a variety of tools such as the ASX Top 200 shares. Other possibilities include CFD’s on stock market indices, foreign exchange and metals. Another way to gain confidence is to enter the annual ASX trading competition. Information about this competition is available through www.asx.com.au. This will help you to be answerable for your trading decisions because you must register your trades with a third party. This has worked well for many traders. Portfolio Research
In order to calculate the amount of risk you are prepared to take and the precise position sizes for your researched trading system, you need to merge combinations of open trades into portfolios. This portfolio is constructed according to the trading system, risk management and money management rules as devised above. During this phase you will actually be researching your maximum drawdown thresholds according to your money management rules. This is where the combination of the trading system expectancy, the risk management rules and the money management rules all come together. Altering your money management rules during this portfolio research phase will lead to different levels of maximum drawdown. Which level you decide will depend on your risk appetite. This will depend on how much drawdown, that is, how much money being given back to the market, you can handle? Did I mention that drawdown cannot be avoided? So don’t try to achieve drawdown of 0%! You might define different maximum drawdown levels which are determined by changing your position sizes. These levels might range from 5% to 50% depending on the markets and terms being traded. Each portfolio should be viewed diagrammatically over time by generating equity curves for each portfolio noticing when the drawdown occurs. These equity curves can be compared to the overall market to measure how well the portfolios have performed in different types of market conditions. When this phase is complete you will be ready to take your methodology to the market. If you would like to follow a trading plan template that asks you specific questions to help you consider all of the issues that are important, you can find one in my book Trading Secrets. Read more articles by Louise Bedford. Read more articles by Stock Investor Journal. Get data to assist you with your trading.