Statistics are the key. It provides a relatively straightforward toolbox to easily dismiss the simplest of systems. The measure in particular that I care about is called a student’s t-test.

Let’s first talk about when this test is appropriate. If you have fixed take profits and stop losses in place on 100% of all trades, then it is appropriate to use this test. The reason is that these values provide fixed limits on the outcome of trades. The distribution of your trades is likely to form a nice bell curve. In math terms, this is called a Gaussian distribution.

If your strategy uses dynamic exits based on market conditions, then this test is not appropriate. The distribution of forex, stock and futures prices do not follow a bell curve. The test depends on the assumption that the distribution under study follows a bell curve. If you make assumptions that don’t match what you’re measuring, then results are useless at best and dangerously misleading at worst.

If you’d like to go into the mathematics involved, then you can find numerous sources on the internet that explain t-tests. I also really enjoy the book Statistics by Freedman, Pisani and Purves.

Most trading systems do not follow bell curves, which makes the details of the t-test largely irrelevant. What’s useful about it is to show that you can generally feel better about the outcomes and predictions based on the number of results in the backtest.

Restrictions and Degrees of Freedoms

The goal of any test is to ensure that the results are accurate. The more frequently that you test a concept, the more confident that you feel about the outcome repeating itself consistently. The idea of predictability largely matches our intuitive expectations. If my co-worker shows up on time regularly, then I feel confident about him showing up on time tomorrow. If he shows up late regularly, then I know that he’s likely to show up late in the future.

The increase in experience increases the level of confidence. Eventually, the number gets so big that we feel very comfortable with the probabilities.

The Portuguese client presented a system based on moving averages with 3 filters, a stop loss and a take profit. Let’s assume that each filter only had one parameter. The number of restrictions for the buy trade is the moving average period (1), the one parameter for each of the three filters (3), the stop loss distance (1) and the take profit distance (1). This yields a total of 6 restrictions for buy trades. Assuming that sell trades use the same inputs, then we have a total of 12 restrictions.

We can’t begin counting our total number of trades (i.e., degrees of freedom) until the backtest shows at least 12 trades to account for our restrictions. It’s a good idea to not infer anything about a trading system until 30+ trades elapse. With the 12 restrictions in place, that sets the threshold for the minimum number of trades to reach a conclusion at 30 + 12 = 42.

Generally, it’s a good idea to see 300-400 trades before drawing conclusions about any system. The reason for this is that some events are very rare. They may only occur once every couple of hundred trials. Allowing the amount of information to approach this threshold allows the trader to more comfortably evaluate what hidden risks may be present.

The backtest that the Portuguese individual submitted only contained 27 trades. Knowing what we know about basic analysis, I comfortably decided that there is nowhere near enough information on the system to consider evaluating it.

A Word of Caution

An algorithm’s trading statistics are almost certain to change with time. Unless you have a mathematically sophisticated model for evaluating volatility, best practice demands that you evaluate the trading results in light of the type of volatility experienced. Making money in 2008 when the markets nose dived does not mean that you would have made money in 2010, which was much quieter in comparison. Any signal in 2008 that indicated a short would almost certainly show returns that include a handful of monster winners. If those monsters fail to show up because the volatility does not cooperate, then your expert advisor will more than likely flop.

If you like this blog, then you’ll love his. It’s very similar. The only difference is that Ernie tends to write with more assumptions about his readers’ math skills.

Files usually start with a declaration of #defines (pronounced pound define) global variables and external variables, also known as an extern data type. They appear near the top of the code to help the read gain an understanding of the variables that will run in the program. Ideally, the names of the variables and how they are organized should assist the programmer with form a general understanding of what the expert advisor or indicator might do.

The next section is usually the init() function, which is the word initialize abbreviated. This section of the code is particularly relevant to programming custom indicators. Most of the general indicator settings like declaring the indicator buffers, the colors to use and other basic features are set within this section. I use init() in every expert advisor that we build to convert the inputs into an appropriate setting for the broker’s pricing. If a client inputs a stop loss of 50 into an EA, I don’t need to do anything if it’s a 4 digit broker. I do, however, need to convert the input to work with a 5 digit broker. I run a quick check within init() to see if Digits == 3 || Digits == 5. If so, then I multiply inputs affected by that setting by 10.

deinit() is the least important section; it’s pretty easy to deinitialize an MQL file because it usually does not take up any system resources. It’s rarely used for anything important. The only uses that I ever have for deinit() are to close an open file handle or to make some sort of closing note. This is often done either on the chart directly through the Comment() function or more often by writing directly into the log file.

The start() function is the real meat of an MQL expert advisor or indicator. Whenever MetaTrader detects an incoming tick, it alerts any MQL programs. Those programs then call the start function so that it can do whatever needs doing. All trading operations, calculations, account monitoring, etc, occur within this section.

All of the other custom functions within the program appear below start(). I usually prefer to rank them in order of their importance or the frequency with which I call them throughout the program. The order of placement of functions does not affect performance in any way at all. It’s strictly a cosmetic practice that makes programming code more legible.

The double data type is probably the most common type found in MQL programs. This is because it is the type responsible for calculating floating point numbers. Say for example, that an expert advisor needs to determine when to adjust a trailing stop. The expert advisor looks at the current price and subtracts it from the current stop loss to maintain the appropriate distance (1.3230-1.3209=0.0021). The distance requires a decimal point. When the expert advisor saves the distance to memory, it needs to save the information after the decimal point. That forces the programmer to choose a variable of type double.

Integers, or int, is the simpler version of double. Double values require a decimal place to hold the number’s value accurately. An integer, or whole number, does not have a decimal place. Integers are appropriately used when the MQL programmer knows for a certain fact that the number will never contain a decimal. An example would be if you wanted to implement a max trades feature. If the number of open trades in the account exceeds a certain number, then prevent trades. We know in advance that there is no such thing at 4.76 trades being open. There can only be 4 trades open or 5 trades open. This clearly indicates the need to use an integer.

Datetime values are just what they sound like. They represent both the date and time. More specifically, a datetime variable represents the number of seconds that have elapsed since January 1, 1970. This is where it gets a little tricky. The number of seconds that have elapsed is actually an integer. Datetimes store integer values but then associate them with a date and time.

A value of 0 would indicate that the time is 00:00 on January 1, 1970. A value of 60 stands for one minute later at 00:01 1/1/1970, and so on. One benefit of knowing that the datetime type stores information as integers is that you can easily determine the amount of time that happens between an event. If the event starts at 15:35 and ends at 18:12, you can simply subtract 18:12 – 15:35 and wind up with the number of seconds between those values. That information can then be used to determine the number minutes/hours/days between the two events.

The color data type, not surprisingly, holds color information such as black, yellow, red and so on. Much like datetime types, color also uses integers to store the information. The difference, though, is that extracting the color information from the integer is not at all obvious. Increasing a color type from 32768 by one will not necessarily make it more or less green. Colors use the integer information to retrieve the red, green and blue components of the color in hexadecimal format. Explaining hexadecimals is well beyond the scope of this article. It’s unlikely to come up in your MQL programming. I’ve been doing this for over five years and only came across one project that required manipulating a color in way more complicated than alternating between two set colors.

A string is anything that resembles a word or sentence. It always uses quotes to contain the information. My favorite use of strings is to gather information to display on the chart or in a log file whenever I need to debug an expert advisor.

Char is the final data type. It’s so closely related to a string that I wasn’t even aware this type existed until I looked up information for this article. If we study the word “trade”, then we will find that it is composed of the five characters t, r, a, d and e.

A final note on data types. There are two ways that types are held in memory. An extern variable is one that shows up in the inputs screen whenever an expert advisor or indicator loads. Static variables are the opposite. They remain within the MQL program and never visible outside of it.

Trading forex with a Martingale money management system will almost inevitably lead to blowing up an account. I’ve written about this inevitable outcomes repeatedly over the past six months. At the risk of beating a dead horse, I figured that visual proof would alleviate any lingering hopes once and for all.

Recall that Martingale systems aim to never lose money. Instead of accepting losses and moving on, a Martingale betting strategy doubles the previous bet. Whenever a win finally does occur, all losses up until that point are wiped out. The trade also gains the same amount of profit that the original trade hoped to capture.

The experiment assumes that the trader uses fixed fractional money management set at 1% of the account value. Recall from earlier experiments that a 1% risk value will almost never blow up an account after 200 trades. The percent accuracy for the trades remains at 50%, which is perfectly random. The random number file has been upgraded to include 10 million random numbers instead of the previous half a million.

The goal of the exercise is to focus on the risk of ruin rather than the profits accrued. As time goes on, the likelihood of ruin increases with the number of trades placed. A trade is each time a new transaction enters. It does not matter whether or not the last trade was a winner or a loser.

Fifty trades on most Martingale systems corresponds to anything from several days to several weeks. The level of aggression used in the trade level (i.e., the pip distance used to open a new trade) is what most strongly affects the amount of time required to reach fifty trades.

Placing 50 trades shows what most traders know. The returns look fairly nice at that point. A return of 20% on the account shows a 40% probability of occurring. The risk of wiping out the account looks meek at 8.5%.

Increasing the number of trades to 200, which corresponds to several weeks or months, the odds of outright failure skyrocket to 35%. The lucky traders that have not yet blown up show returns ranging from 20% all the way to 300%. The total risk is more apparent, although many traders fall victim to the lure of quick, large returns.  If it all looks too easy at this point in time, that’s because it is.

Going out to 1,000 trades, which I roughly ballpark as the amount of trades an average expert advisor might complete in 9 months to a year, is where the inevitable result is obvious. The odds of reaching a zero balance reach 95%.  A tiny handful of traders are floating huge returns. As the number of trades increases from 1,000 to 2,000 to 10,000, the tiny fraction of accounts left eventually dwindles down to zero.

Stop and Limit Entries to Catch Trends

Many trading systems use market orders to enter and exit the market. Market orders frequently exhibit low entry efficiencies. They get you in the market at a price that is not optimal. Placing a buy or sell order 10 pips from the price you wish to enter, in the direction of the trend for trending systems can make a big difference. For long trades put a buy order 10 pips higher than price, and for short trades put a sell order 10 pips lower than price. Range trading systems might consider limit entries, which would place the orders in the opposite direction of the example above.

Using ATR to Account for Volatility

Many novice system designers use constant pip distances in their forex trading system, i.e. 15 pips for stop loss, 10 pips for take profit, etc. This is a mistake as it doesn’t take into account changes in volatility. If the pair you trade exhibits changes in volatility, the system faces an increased likelihood of failure.

The Average True Range indicator (a.k.a. ATR) gives the average range of a forex pair or stock, and accounts for gaps as well. Instead of using constant numbers, use a percentage of ATR such as 50% ATR or 30% ATR. Once you do this change your system will automatically take into account volatility and will become much more flexible. Such systems will work better and will maintain profitability even in changing market environments.

Avoid Overoptimization

This is a tip especially for the programmers of you: over-optimization is the kiss of death of a trading system. Over-optimization, a.k.a. as curve fitting, means that you add many Forex indicators and filters and use them all to confirm your signals, and optimize them all for maximum profits. In the backtest it will seem that your system is becoming better when in fact it will become good only for the past and will fail in any forward-test on real, live data. Therefore, it is crucial to only include the most important parts of your system and do not add indicators that don’t make sense at the price-action level. Remember the principle of Occam’s razor: “The simplest solution is usually the most efficient one”.

Michael Wells is an FX programmer and trader. His site contains his insights about Forex trading systems.

The launch mainly signifies that the larger forex firms will start rolling out their own installations of MT5 within the next six to twelve months. Rumor has it that Alpari’s owners are very close to the owners of MetaQuotes. Perhaps this is hearsay, but it’s my impression that Alpari is the first among equals when it comes to MetaQuotes’ clientele. Alpari did pay up the wazoo, however, for their license. Maybe they’re just getting rewarded for adopting the new platform so quickly.

Most brokerages, especially the large ones, are not chomping at the bit to adopt the new release. In fact, most of them hate MetaTrader with a passion. The back office is written largely for brokerages that exclusively want to use MetaTrader. The larger brokers, all of whom invariably offer their own proprietary platforms, have to jump through a lot of hoops to get all the moving parts between separate back office systems working in sync. The rollout will likely embroil their IT staff in problems for months on end. I seriously doubt most CEOs are looking forward to the switch.

Also, offering MT5 as the primary platform does not mean that your brokerage is going to flip the off switch on MT4. They depend on MetaTrader 4 for their cash flow. Brokerages will not sabotage themselves by preventing all of their customers from trading.

Rollouts of new technologies usually occur over a period of 9 months or more. When I worked with FXCM, I remember the handful of clients that refused to switch from Trading Station I to II. It wasn’t until 2 years after the initial release of the new version where the company decided to drag the stragglers kicking and screaming onto version II.

The switch from MetaTrader 3 to 4 worked in much the same way at the brokerages offering it at the time. It wasn’t until two years or so after its initial adoption that version 3 went by the wayside.

You have little to worry about as a retail trader considering the switch over to MT5. If you want to program a brand new EA and your broker already supports MetaTrader 5, then you should definitely program it in MQL5. Otherwise, stick with MetaTrader 4. It still has years of shelf life.

Fixed fractional money management changes the overall outcome of your trades. Remember that trading is the net outcome of several hundred trades.  The power of money management comes into play after several hundred trades or more.

Trading totally at random with a 50% winning percentage and an R multiple of 1 yields no advantage, as I discussed last week in modeling money management. Remember that an R multiple is the average win to the average loss.  Such a system poses neither an advantage or disadvantage. The average outcome should come out extremely close to the starting balance.

Fixed fractional money management stretches some portions of the bell curve and compresses other regions. Before we get into that, it’s important to remember what fixed fractional money management means. It’s a complicated name for a simple concept. It stands for the idea of risking a set percentage of the current account equity rather than the starting equity.

Most traders focus on risking a set dollar amount such as $1,000 on a given trade. This method updates that dollar figure after every single trade.

Consider an example where the account balance starts at $100,000 risking 1%. Both methods risk the same amount on the first trade, $1,000. The next trade, however, will yield a different risk amount. A win on the previous trade would increase the account equity to $101,000. One percent of a 101 grand is $1,010 of risk on the next trade. A whopping ten dollar change.

That may seem trivial. It is most certainly not over the long run.

Examples

Consider a trader that plays the coin toss game and has a system with the following characteristics:

• He starts with a $100, 000 account balance
• His R multiple is 1.0
• He wins 50% of the time with no trading costs
• He risks 1%

The absolute worst outcome of playing the coin toss with a fixed dollar risk of $1,000 is a loss of $46,000.  Adding fixed fractional money management during that difficult drawdown improves the drawdown to a less substantial loss of $37,500. The worst drawdown goes from -46% to -37.5%. The method drags the absolute worst case scenario and pulls it closer to the average. When an unlucky, devastating drawdown kicks in, the technique reduces the losses that the trader experiences.

The best case scenario for fixed dollar risk is a $58,000 (58%)  return.  Adding money management to the system dramatically stretches the best case scenario further to the right. It improves to a $76,000 return (76%).  The good times get a lot better without changing anything at all about the trading system. The method stretches positive returns away from the average. The trader walks away with more money in his pocket.

The natural instinct is to conclude that fixed fractional money management is the way to go. I agree. It improves the risk reward profile of a totally random strategy. Adding it to a real trading system should help control parameters that most traders consider critical like drawdowns and maximizing the return.

An important consequence of using fixed fractional money management, however, is that the odds of receiving a below average return increase somewhat. The coin toss game suffered a below average return 47% of the time. Applying fixed fractional money management increased the likelihood of a below average return to 53%.  The effect is not all that much. Losing is more likely. But when it happens, the “loss” is so negligible that it can be thought of as breaking even.

Random numbers occasionally follow a seemingly non-random pattern such as loss-win-loss-win. When this occurs, the size of the trade on the losses is bigger than the trade size of the winners. Even if the winning percentage comes out at precisely 50%, those wins get slightly overshadowed by the losers.  That micro effect of slightly larger losses than gains shows up as a slightly increased risk of not making as much money as expected.

Graphing all outcomes

Fixed fractional money management changes the shape of the returns curve. This example is exaggerated to highlight the effect

Red areas represent the losing outcomes while green areas represent the winners. Money management is really about maximizing the ratio of green area to red area. Random trades with no expectation of profit yield a bell curve, which appears on the left.

Fixed fractional moves the highest density of returns slightly to the left. Doing so creates the trivial disadvantage of a slightly increased risk of negligible loss. Importantly, the far left side (the worst case loser) gets dragged much closer to the average. The far right side (the best case winner), gets stretched much further from the average. The green area is now larger than the red area. The random system has a minor expectation of generating a positive return!

We recently developed software to model the affects of random chance on money management. Although computers are capable of generating pseudo-random numbers, the pseudo-random process introduces bias into the random distribution.

We opted to source our random numbers from random.org. The web site generates purely random streams of bits taken from atmospheric noise. We then use binary mathematics to change those bits into numbers ranging from 1-10,000.  Say, for example, that we want to model a trading system with a winning percentage of 50%.  Whenever a number comes out between 1-5,000, we consider that a winner. Anything above 5,000 marks a loser.

Changing the winning percentage to 65% works the same way. Any number less than 6,500 represents a winning trade. Numbers above 6,500 signal a loss. The modeling quality is accurate to the thousandth decimal place.  That type of accuracy is way more accurate than the “known” accuracy of your trading system, which can only be known within a few whole percentage points.

Most traders fall into the trap of thinking about their trades as individual outcomes. The more appropriate way to view returns is as the sum of all individual outcomes. Losing on any given trade does not matter. It only matters whether the sum of all your winners is greater than all of the losers.

It gets more complicated, unfortunately. A system with 50% wins and a 1:1 payout will almost never come out at exactly breakeven. The mathematical expectation is that we expect to see a degree of drift in the returns solely due to random chance. I suggest reading more in the random trade outcomes and dollar profits section to get a better understanding of drift.

Lastly, we must define a sampling period for evaluating the final result. I arbitrarily set the default value to 200. That means that the software tells us the range of outcomes after 200 total trades. That may take more than a year for some traders. Daytraders may reach that benchmark in several weeks of trading.  The question that we are answering is “what is my account balance likely to be after placing 200 trades?”

Coin Toss Trading Experiment

The first experiment is to analyze how dollar returns vary with a coin toss game and the most basic money management method. A starting balance of $100,000 is used with a risk of 1%. The risk will not change as in the fixed fractional method. Instead, we will leave the lots fixed in order to strictly understand random chance. Wins always earn $1,000. Losses always lose $1,000. The odds of a coin toss are 50% wins with 50% losses and a 1:1 reward risk ratio.

The average trade comes out to $99,868.36, almost exactly $100,000. It’s what we expect for a 50-50 game with a 1:1 reward risk ratio. What I find interesting is the standard deviation of $14,377 and how it changes. I don’t want to cover scary math topics. The layman’s explanation is that the standard deviation is the “normal” range from the average that you might expect. The $100,000 balance, in most cases, would either lose $14,000 or make $14,000.

Everything beyond those standard deviation boundaries represent the less likely wild scenarios. The minimum outcome comes out to $58,o00, a massive 42% loss. This had a 0.54% chance of occuring. The maximum outcome shows as $158,000, a monster 58% return. This had an even smaller chance of occurring, only 0.1% (1 in every 1,000 trials).

Changing the account risk dramatically affects the standard deviation. 1% strikes most traders as sane and reasonable. Yet, there is a small chance of losing half the account to drawdown strictly because of terrible luck. Decreasing the overall risk by a fourth to 0.25%  drops the standard deviation by exactly one fourth.  The worst case scenario shrinks to a highly tolerable $11,500 drawdown (11.5%). Most traders would find a number between 10%-20% reasonable. The consequence of the reduced risk is that the best case scenario drops correspondingly down to a 14.5% gain.

Stretching the risk out to 2%, a normal industry practice, turns out to be risking accounting suicide with the coin toss game.  The worst case scenario drops the final account balance down to $8,000, a staggering 92% loss.

The goal is to help you define risk from a gut feeling matter into something more tangible and calculated. Too many traders enter the market day dreaming about profits. Risk enters the picture, but too few traders actually understand the relationship between risk and reward.  Hopefully, the picture of best, worst and average scenarios is starting to become more clear for you.

FXDD offers historical price data for all the major currency pairs. The prices extend all the way back to 2005. This option is far and away the easiest to use. The files download in .hst format, which means that you don’t have to worry about formatting. The data resolution is at the M1 level. http://www.fxdd.com/en/mt1m-data.html

Alpari used to offer M1 data on its web site. They fluctuated between taking the links down and putting it back up eventually. Their download links, however, have been down for about two years now. It looks like they finally threw in the towel on offering free price histories.

FXCM offers the ability to export historical prices through their Strategy Trader platform. Obtaining the data is simple. The formatting is a little off, so you would need to write a simple program to convert this into an .hst file. http://www.fxcm.com/strategy-trader.jsp

GAIN offers free data, but they seem to make it intentionally difficult to find and work with. They split the data into one week intervals, and the price histories don’t appear to be even based on my quick glance over their csv file. I wouldn’t recommend going to all the trouble of downloading and formatting their data without a compelling reason. http://ratedata.gaincapital.com/

Dukascopy offers extensive price history down to the tick level. If you want tick prices going back quite a way back in time without paying for data, this is the only option that I know of. I have not messed around with it personally. I only know about it from internet chatter for people who seem to feel that they need tick data for their backtesting. The formatting is not designed for MetaTrader. Once again, you would have to write a conversion program to move it into the MetaTrader history center. http://www.dukascopy.com/swiss/english/data_feed/csv_data_export/