profit factor Archives - Algorithmic and Mechanical Forex Strategies

The guy that bet on Leicester City every year

September 5, 2016 by Shaun Overton Leave a Comment

Leicester City started the 2015 season with terrible odds of winning the Premier League Championship. Bookmakers only game them odds of 5,000:1 of winning.

To put that in context, you are more likely to die riding a bicycle than you were to win a bet on Leicester City. Or, you can think of betting on Leicester City every year. If you bet on them every single year for 5,000 years, you would expect them to win a grand total of… once.

2014 was hardly an indicator of their pending success. They were nearly relegated to a lower division (i.e., kicked out of the Premier League). And yet, they did win the championship last year.

Leicester City’s Biggest Fan

Meet John Michklethwait. He’s the former editor-in-cheif at The Economist and he’s currently editor-in-chief for Bloomberg. Clearly, he’s a very smart man. And yet, despite the odds and repeated disappointments, John bet on his old love, Leicester City, every single year dating back to the 1980s. That’s roughly 30 years of nonstop losing.

It wasn’t a lot of money each year: just £20. We all have our indulgences. I see the value of having skin in the game. £20 on a season is enough to make one care, but not so much that he’s upset about losing it.

Then something disruptive happened. John moved to the US last year for his position at Bloomberg. The chaos of the move threw him out of sorts, and he accidentally forgot to bet on Leicester City in 2015. He bet on them every single year dating back nearly 30 years. And yet the one year that he forgets to bet, not only did Leicester City win, but the bet paid out 5,000:1.

Let’s step back and calculate the cost of that oversight for Mr. Micklethwait.

£20 * 5,000 = £100,000.

A hundred… thousand… pounds. That kind of winning would put a nice dent in your mortgage, wouldn’t it?

The risk of low probability strategies

Everyone hears anecdotes about successful trend traders. Even winning only 30-40% of the time, they walk away big winners over time.

You live HERE. Math isn’t good enough. You also need to wonder if your strategy can handle real-world problems.

What if they took that even lower? They could move their stop losses closer to the market. They’d reduce the size of the average loser, but the winning percentage might also drop to 10-20%.

Mathematically, this could work out identically. 30% winners that earn 5x the average loser make for a profit factor of 1.5. A strategy with only 10% winners that make 15x the typical loser also have a 1.5 profit factor.

They’re the same. Aren’t they?

Planet Earth isn’t the same as planet Math. In the real world, people get sick and miss trades. Or, they move across the Atlantic and forget to place a £20 bet.

People move. They get sick. Computers break. Things can and will go wrong with trading.

Richard Dennis once commented that the Turtle Traders would often make their annual returns off of one, single trade. A single trade!

When your performance depends on positive outliers, you’re massively vulnerable to accidents. What happens if you’re sick that day? Or your internet goes down? Or your broker locks you out of your account on the worst possible day?

Life happens, brother. A plan that depends on perfection is no plan at all. You need to make yourself robust to failure. Or even better, you’d make yourself antifragile.

Winning percentages

I mentioned that you can do really well winning 30-40% of time. Why then, does my own trading strategy, Dominari, win 68% of the time?

Because I’m exploiting compound, exponential growth. It’s not just how much you win, but the order in which you win it.

Let’s take two examples:

A ranging strategy with a profit factor of 1.3 that wins 68% of the time.
A trending strategy with a profit factor of 1.3 that wins 30% of the time.

Look at the red circles. Trending strategies are prone to extreme outcomes, both positive and negative.

Each strategy risks about 1% on any given trade. And, the average of the range and trend strategies are identical in the long run.

But… and this is an important “but”, the expected worst case scenario with the trending strategy is substantially more likely compared to the range trading strategy. In effect, the average is more average with a ranging strategy than with a trending strategy.

Why is that? Because unusual losing streaks are devastating to trending strategies. Have you ever had a losing streak? It happens to everyone.

By using a strategy with a higher winning percentage, you’re making yourself robust to streaks of losers. And, not to mention, your average length of a winning streak is considerably higher.

Even though you’re getting the same mathematical outcome, you’re making things much easier on your trading psychology when you adopt a strategy with a higher winning percentage.

Dominari & Exponential Growth

You may have thought to yourself, “68%? That’s kind of a strange number to pick.”

You’d be right. The choice of 68% winners was not a coincidence. It is, in fact, the win rate on my Dominari strategy.

Dominari is about more than just buying and selling. Trading is also about managing a portfolio and position sizing. Position sizing is phenomenally important over your trading career.

My backtest results for Dominari show that for every $2,500, the account increased to $17,855.35 after 3 years. That kind of compound growth doesn’t happen by accident. That’s why I’d like to share the good news with you in my webinar this week.

I’m going to show you how to put that exponential awesomeness to work in your trading account. Sound good? Click here to register for the FREE webinar.

Big change to Dominari

March 9, 2016 by Shaun Overton 24 Comments

I said it here and here and here. The biggest issue with my Dominari is trading costs. Things aren’t going to really take off until I do one of two things.

Reduce the trading costs
Make more money on each trade

I’ve been working on Dominari since around September or October of last year. After racking my brain for months, I more or less wrote off the idea of improving the trade profitability.

That suddenly changed last week on Friday after the market closed. The best reason to trade my own systems live is that the agony of underperforming forces creativity. The feeling reminds me a lot of Daymond John’s (the guy from Shark Tank) new book the Power of Broke. When life isn’t going your way, it’s the resourceful and creative who are best able to get to the top.

Nobody wants to feel broke or under extreme stress. As much as we hate those feelings, they’re often the strongest drivers of performance. That’s how I feel right now with Dominari. I’m so close to getting there and wasn’t sure how to fix that missing ingredient.

If it weren’t for that stress, I would not have had my simple but very powerful insight last Friday.

And please don’t laugh. The change is so dumb and obvious that you’re going to wonder what’s wrong with me. When you’re in the thick of designing a system, the ugly truth is that sometimes you get lost in the weeds. Or to use another botany metaphor, you only see the trees instead of the forest.

My key insight was to slightly modify the exit strategy to use limit orders, whereas previously I only exited based on the close of the bar. I noticed two repeated behaviors that finally beat me over the head enough that the point finally sank in.

The number of occasions where my trade closed in the optimal location seemed to be significantly outweighed by the amount of money left on the table. The key insight for me was realizing where to optimally place that limit order. And for those of you on my newsletter, it happens to be closely related to the Auto Take Profit that I’ve been talking about all week.

Backtest assumptions and results

My operating mantra when doing backtests is to minimize the number of assumptions. Spreads for retail traders have changed dramatically from 2008 to today. I remember working as a broker at FXCM when our typical spread on GBPCHF was something like 8-9 pips. I now routinely pay something like 2 pips. It’s impossible to model what happened in the middle without haphazardly guessing.

I find it far more convincing to analyze the raw signal, both on historical and recent market data, then to interpret whether trading costs are likely to be favorable in today’s markets. “Raw signal” is the ideal signal, one which assumes perfect execution, no slippage, no rollover, no spreads and no commissions. The natural result is that you’re overstating historical performance, but the benefit is that you have a very clear idea whether the core idea is a system capable of predicting the market with reasonable risks.

The total leverage employed in the portfolio is 7:1. If I have a $50,000 trading account and held a position in every currency pair in the portfolio, then the notional value of those trades would equal $350,000 (50k * 7).

Another key point is that I used a fixed position size of $12,500 per trade. The size of the trade never increases or decreases during the backtest, which allows me to isolate the impact of the raw signal without adding the variable of money management.

Here are my trade metrics with version 1 of Dominari. Click the images to view them in full size.

The first version of Dominari had a profit factor of 1.26.

After here’s the change with Dominari version 2.0.

My new version of Dominari increases the profit factor to 1.59 with a significantly lower drawdown.

My best case scenario was to hope that the profit factor would jump another 10 points or thereabouts, maybe stretching the profit factor to 1.35 or thereabouts. It’s incredibly exciting to see the edge over breakeven more than double (going from a $0.26 edge to a $0.59 cent edge).

What I’m most excited about is the skew in the returns. Most mean reversion systems look for an edge but are overwhelmed with the impact of losing trades. That was the case with version 1.

The largest losers outweighed the largest winners in version 1.

This new version of Dominari is the very first mean reversion strategy that I’ve ever developed where the winning tails (ie, the biggest winners) nearly equal the losing tails (the biggest losers). It’s almost always the opposite with mean reversion strategies. Said another way, the risk profile of the extreme outcomes significantly improved with version 2.

The impact of the biggest winners is nearly identical to the biggest losers with version 2.

And the metric that most traders care about the most, drawdown, is wildly improved. Version 1 showed a drawdown of 5.72%. The new version is a fraction of that at 1.77%.

Out of sample backtest for Dominari version 2

The out of sample performance is nearly identical to the in sample performance, despite significantly different market conditions.

When I walked my test out of sample onto recent data, covering 2013-2015, the performance characteristics of version 2 are nearly identical to the in-sample test. The profit factor was identical at 1.59, and the max drawdown was 2.01% for 2013-2015.

Translating the theoretical into expected performance parameters

Again, those metrics above are in the ideal world of perfect execution and no trading costs. The real world performance will have lower returns and higher drawdowns. The advantage to having live trade data is that I can now make some kind of intelligent estimate of my expected trade accuracy and profit factor. Just how overstated are the idealized returns likely to be?

The process that I went through to calculate the expected profit factor in the real world is a 5 step process. I don’t think it’s going to make any sense if I try to write out the steps in conversational English. Instead, I’ve chosen to share a spreadsheet where you can view the step by step process for how extrapolating live trading data into expected performance with the new strategy. Click here to view the spreadsheet.

The expected profit factor for my live trading is expected to be between 1.29 to 1.39. The expected percent accuracy for live trades should jump from 62.55% to 70.8%.

The traders who will get first crack at the Total Access Apprenticeship are those are subscribed to the free newsletter. If you’re not signed up, make sure to fill in your email address in the orange box at the top right of this page.

Money Management for High Probability Systems

March 12, 2013 by Shaun Overton 8 Comments

I spent last weekend overviewing trading systems with my favorite money management software. Normally, when traders think of a great trading system, they expect fantastic results like 80% winners and every trade has to make $2 for every $1 that it loses.

The edge in real trading systems typically falls wildly short of that goal post. Rather than convince you with software, try running the numbers in your head.

Say that we do 100 trades with a $100 account balance. 80% of them win and make $2, which is $160 in profit. 20% of them lose $1, which is $20 in total losses. Now ask yourself, “Does it sound plausible to earn a 40% return after only 100 trades?”

The answer to the question probably correlates with your trading experience. If you answered yes, then I also have some oceanfront property to sell you in Arizona. Experience should make it clear that such numbers are fairy-tale optimistic.

Las Vegas makes billions of dollars every year with a total edge of 0.5%. You don’t need to hit grand slams to take money out of the markets. Little base hits will get you there, too.

What is high-probability

At the risk of moving the goal posts, I define high probability as anything above 50%. Many traders comes across the word “high probability” and start drooling over 80-90% winners.

Remember that winning percentage isn’t everything. If we play a game where you win 99% of the time but lose $200 when you lose, you’d quickly figure out to not play. The name of the game is to walk away with a profit over time. It doesn’t matter how you get there.

A high probability, one which exceeds 50%, offers the unique advantage that you can aggressively use money management to enhance returns. High probabilities also come with the unique property of offering long streaks of consecutive winners. Consecutive losing streaks are comparatively rare.

Money management in a real strategy

One of my clients trades a strategy with 60% winners. The ratio of the average winner to the average loser. That is, the average winner earns $1. The average loser loses $1.

Because it wins 60% of the time, that means it makes $60 for every $40 it loses, which is a profit factor of 1.5. Not bad.

A strategy with this property expects to earn a return of 40.9% after 200 trades if it risked 1% of the original account balance per trade.

The strategy using fixed dollar risk expects to make a 40.9% average return after 200 trades.

Making a minor change so that the strategy risks 1% of the current balance per trade makes a huge difference in the average return. It jumps nearly 10% to an average return of 50.7%.

Fixed fractional money management is the idea of keeping the risk proportional to the current balance. Using a $10,000 account as an example, a trade risking 1% of the balance risks $100. If the account balance increased to $11,000, each trade now risks $110. The dollar risk grows with time while the percentage risk remains unchanged – hence, it’s fixed fractional.

$Fixed fractional money management$

A fixed fractional money management approach increases the average profit to 50%.

A unique money management idea

Remember how I mentioned that high probability systems are prone to streaks of winners and losers? Check out what happens if you increase the risk after each winning trade by 10%.

The median outcome gets pulled up enormously. The average isn’t jumping to 63% because of a handful of massive returns. Instead, almost every series in the test experiences a huge increase in performance.

The average balance after 200 trades jumps yet another 12% for a 63% return. The only disadvantage is that the worst case scenario dropped from a 6.8% loss to a 10.3% loss.

Position sizing for consecutive winning trades

Increasing the risk by 10% from the previous trade increases the average return to 63% and the median return to 58%.

Applying the rule so that consecutive winners only increase their risk when the balance is greater than the starting balance makes the worst case scenario slightly less bad. It comes at the price of a slightly lower average and median return.

Consecutive winning trades applied only to a profitable balance

Applying the consecutive winning trades rule only slightly helps the worst case scenario. It comes at the expense of a lower average and median return.

Conclusion

You’d be a fool to stick with fixed risk if your system offers offers high probability winners without huge losing trades. Simple steps like fixed fractional money management can substantially increase profits. Applying a trick like increasing the risk by 10% after each consecutive winner offers another huge performance boost.

The average outcome started at a healthy 41%, which is already stellar. Applying the simple techniques outlined here increased the average performance after 200 trades to a 63% return. That’s some world class trading.

Random Trailing Limit

March 12, 2012 by Shaun Overton 14 Comments

I got the idea for a trailing limit from Van Tharpe’s market classic Trade Your Way to Financial Freedom. I made Jon Rackley read through the book as part of his training when I first hired him. He brought my attention to an unusual claim made on page 267. Van Tharpe says that it’s often possible to make money even with random numbers.

Random numbers are a pet theory of mine. I’ve long put effort into figuring out whether I could develop a strategy that trades totally at random and still makes money. As the owner of a programming company for traders, and as part of Jon’s initial training for NinjaTrader in December, I assigned him the task of programming the strategy into code.

The book states that trading with totally random entries and a 3 ATR trailing stop generally leads to making money. Flip a coin. You go long if it lands on heads. Go short if it lands on tails. One important note is that we elected to use pure random numbers in our programming instead of the pseudo-random numbers that computers generate. Doing so allows us to avoid time biases in the seeding process that generally pop up when the seeds used are close together in time.

The first working version that I reviewed displayed everything contrary to Van Tharpe’s claims. Using a trailing stop, regardless of the instrument and time frame tested, inevitably led to devastating losses. The profit factors consistently came in near 0.7, a truly awful number.

We did what most of our clients do when they find abysmal strategies. We flipped the strategy on its head. The new strategy uses only a 3 ATR (50 period) trailing limit.

Trailing Limit Analysis

The early conclusion is that trailing limits seem to offer a great deal of potential. Although Jon sent me the code several months ago, it’s only this evening that I had a chance to properly review and test everything.

The profit factors are very encouraging. It depends on the chart that I tested. The worst that I found was a 1.0 profit factor. Everything else came out with profit factors greater than 1. The small table below contains the initial test results. Before you go off salivating that this is the next hot winner, there are a few considerations to keep in mind:

The results depend entirely on the sequence of random numbers used. Using different sets of random numbers will cause different outcomes.
The better results on the higher time frames likely result from sampling error. The number of trades involved was only ~160, which is not enough to make definitive conclusions on the nature of the performance. I prefer to see 400 or more trades before drawing definitive conclusions.
These results do not include spread costs or commissions.

Currency	Period	Profit Factor	Dates Tested
EURUSD	M1	1.12	9/12/2011-3/12/2012
GBPUSD	M1	1.16	9/12/2011-3/12/2012
USDJPY	M1	1.25	9/12/2011-3/12/2012
EURUSD	M5	1.11	2011
GBPUSD	M5	1.0	2011
USDJPY	M5	1.06	2011
EURUSD	H1	1.26	2011
GBPUSD	H1	1.25	2011
USDJPY	H1	1.48	2011

Random trades sometimes produce solid looking equity curves. Remember that this was generated with purely random numbers and a trailing limit

What made me feel better about the results was reviewing the entry and exit efficiencies of each strategy. The number of trades involved really cluttered the graphic. I ran a backtest on a much shorter time period so that the horizontal, blue line would appear clearly on each graph.

The entry efficiency of a random entry is… random (45.45% entry efficiency)

The exit efficiencies of random entries with traililng limits are outstanding

The exit efficiencies of random entries with trailing limits is off the charts. (77.73% exit efficiency)

The entry efficiency tells us exactly what we would expect to find. Trades which enter at random do not perform better than random (obviously). What’s interesting is how the trailing limit exit strategy actually skews the entry efficiencies to read slightly worse than random. This is a good example of why it’s dangerous to rely purely on statistics. Keeping the big picture in mind reduces the likelihood of making an erroneous conclusion, in this case that there might be something “wrong” with our perfectly random entries.

The exit strategy, which is what we’re truly testing, looks absolutely stellar. It shows that acting as a conditional probability allows for a great deal of adverse movement while capturing extreme points of the move.

Adding Money Management

The percentage of winners for the tested charts ranges from 60-67% accuracy. High percentages of winners often lead to winning streaks. My cursory glance through the chart led to my easily finding a suitable example to cherry pick. Here, 5 trades in a row reach their near maximum take profits.

Picking trades at random occasionally leads to streaks of lucky winners

Testing that I’ve done in the past tells me that it’s often advantageous to increase the position size based on consecutive winners when a strategy is more than 50% accurate. My first idea after reviewing the initial results was to modify the forex money management strategy to pursue the consecutive winners. We unfortunately do not have time to pursue those changes in the code right now. Let me know if you’re interested in seeing this and we’ll make it a priority if enough readers respond.

While increasing the risk after consecutive winners works out statistically in your favor, it does add to the risk. It’s entirely possible for a “winning” set of trades to start losing based solely on the money management strategy. There’s no way to know whether your set of trades will be the luck beneficiary of the extra risk or whether it will be the unlikely loser.

Conclusion

Everyone talks about stops. You always have to have a stop. Blah blah blah. I know it’s going to be the first question that everyone asks.

My research shows that trading with a stop is for suckers. Larry Connors’ book Short Term Trading Strategies That Work was the first trading book where I actually felt like I read valuable information. One of my favorite sections is on stop losses. His research shows that using a stop loss (even a 50% stop loss) always reduces the performance of a strategy. My own independent analysis bears this out.

Not using stops does not mean never taking losses. On the contrary, refusing to exit the market at a loss makes for a 100% odds of blowing up one day. The point of using a stop or limit is to empirically define, and to never waver from, a specific point or points in the market at which you will exit. A trailing limit accomplishes that goal admirably.

Interestingly, the trailing limit is one feature which FAP Turbo incorporates that I have yet to find in any other expert advisors. Although I’m not a fan of FAP Turbo type of strategies, it certainly does interest me that one of its main features aligns with this research. Also notable is the fact that the trailing limit continues down even to the point where it accepts losses.

Range Trade at High Frequency

February 28, 2012 by Shaun Overton Leave a Comment

Range trading systems make the best candidates for high frequency systems. They are less execution sensitive than trending systems for a simple reason. Range trades “catch the falling knife,” making them suitable for using limit orders.

High frequency prices vary from the normal M30 and H1 charts. The lower the time frame, the better that the chart fits to a normal bell curve. One common theme in systems trading since the 2008 crash has been “tail risk” or “fat tails”, which refer to the edges of a probability distribution like the bell curve. The fatter the tails, the more likely that a range trading system is to crash and burn.

The high frequency bell curve shows the tail risk of important events

The bell curve shows the tail risk of important events. The tails are colored in red. Fat tails mean that important news happens more frequently

The real world events captured in the tails reflect headline news like Bernanke speaking or Ireland announcing another referendum on all this bailout nonsense. The events only happen once, obviously. If you consider the news events in the context of hourly charts, they happen frequently as a percentage of the overall period. If you look at a one minute chart, that same event is now about 1/60th as important. Dropping down to tick charts nearly makes the events disappear in the statistical profile.

My experience is that the news cycle drives trends on a macro basis. “Macro basis” and high frequency are two topics that don’t belong together. Trending systems should focus on long term trading, while ranging systems are far more suited to high frequency. If your system trend trades, you can throw it in the rubbish bin for high frequency trading ideas.

High frequency considerations

Keep in mind that there are effectively two ways to participate in the forex market: you can either act as a price taker or as a price marker. Price takers range across all market participants. A hedge fund or university endowment is just as likely to take a price as they are to make one. CTAs and retail forex traders are much more likely to make their decisions based on the expected market direction. Timing is critical for them, so they don’t want to leave it to chance whether or not they’ll get to enter a trade.

The trader gets filled right away. That’s the major advantage. The main disadvantage to acting as a price taker is that you pay the spread every single time that you want to enter a position.

I sat with AvaFX in Dublin on my last trip. They charge a 3 pip fixed cost spread. I mentioned my concern about how that spread affects my client’s EA performance. His MetaTrader expert advisor trades 4 times per day on 2 currency pairs. If you do the math on a 3 pip spread, it works out to 8 * 260 = 2,080 trades per year. If you’re paying 3 pips and trading a $10,00 account, you would have to earn $6,240 per year – a 62.4% return, just to cover trading costs. I don’t care how good a system is – it will never cover those kinds of costs. Trading on margin will not do anything to resolve the issue. Spread costs are directly proportional to the amount traded, which impacts the profit. There is no way to trade and make money if the transaction costs are too high.

Designing an expert advisor is difficult enough, but it’s even harder when you factor in the trading costs. Say, for example, that I develop a EA that wins 75% of the time with a payout of 0.5:1 before trading costs. When the EA wins, it earns $0.5. It loses $1 whenever a loss occurs. The profit is 75 wins * $0.5 = $37.5. The loss is 25 * $1 = $25. The expert advisor’s profit factor is 37.5/25 = 1.5.

That should sound great. The problem occurs when the total commission outweighs the total expected profit. This example required 100 trades. Let’s say that we were trading mini lots with an average win of 5 pips and the average loss of 10 pips. That puts the gross profit at $375 and the gross loss at $250. The return is $125 for the 100 trades, excpet that we must now subtract the $100 for trading costs. The total profit plummets to a measly $25.

If the expert advisor’s expectations held true for something like a 10 pip take profit and 20 pip stop loss, the trader might be better off to change the exit points. The reason is that the profitability may actually improve. The goal would be to reduce the number of trading opportunities with an eye towards making them more profitable relative to the costs.

A better approach, in my opinion, would be to switch over to market making. Although you usually still pay to trade, the advantage to market making is that you earn the spread rather than paying it. The spread is overwhelmingly most traders biggest cost. Not paying it opens the possibility of applying the strategy where one normally could not afford it.

Market making only works if your forex broker allows you to post best bid/best offer and have the price reflected on the screen. Most brokers claim that they are ECNs. A real forex ECN allows you to post limit orders. Whenever that order represents the best bid or offer, the price and size of your order shows up on the screen. The only retail trader friendly brokers that I know of are Interactive Brokers and MB Trading.

I ran my NinjaTrader license at MB Trading last week to test the execution and order flow. The test only use traded a microlot (0.01) and posted best bid or best offer on the EURUSD. The orders remained valid for anywhere from 1-10 minutes. Despite the small trade size and lengthy time period as best bid/offer, the orders only filled 75% of the time. That meant that I caught 100% of the losers but only 56% of the potential winners. Not good, in spite of getting paid for the limit orders.

Interactive Brokers is the next test candidate. They have been around much longer and should have far more order flow. I’m hoping that the low fill rate that I experienced making a market at MB Trading will improve substantially when I shift the same strategy to Interactive Brokers.

I expect to find a few other changes as well. The spread that I earn should fall from around 0.9 pips on EURUSD to 0.5 pips, which is indicative of Interactive Brokers’ improved pricing. I also will have to pay a 0.2 pip commission, which reduces the net credit from 1.0 pips at MB Trading (0.9 spread + 0.1 commission) to 0.3. Nonetheless, I expect the improved fill rate on winning trades to work more in my favor.

The thing that most people will hate is that you can only test a market making approach with live money. It’s sufficient to backtest a strategy using market orders with a 0 spread assumption. The goal is to weed out the junk from diamonds in the rough. No method exists, however, to accurately determine whether or not a trade would have gotten filled with a limit order. The only way to find out is to test an idea with live money, then to compare the results to a backtest over the same period. If the live, high frequency performance is similar to a backtest, then you probably have a winning approach.

The real motivation here is to get as many opportunities as possible. Just like the casino does everything to help you pull the slot machine faster, the trader should look for as many favorable setups as possible. High frequency stands out in this area. The inherent advantages of a system are more likely to manifest more quickly. Assuming that you get a handle on the trading cost problem, the profit is often limited only by the number of trades that can be squeezed into a day.

Programming options at high frequency

MetaTrader 4 is not a good candidate unless you expect to post orders once per minute or slower. MetaTrader suffers from the Trade Context is Busy error. Running an expert advisor on more than a single instrument could cause orders to enter too slowly or not at all. MetaTrader is only an option with MB Trading. Interactive Brokers does not support MetaTrader.

NinjaTrader works great and offers a lot of the broker portability that comes with programming in MQL. Programming a high frequency strategy in NinjaTrader works at most human speeds (5 seconds or more). For the brokerages where NinjaTrader submits orders using the broker’s API, I find a speed bump affect at work. NinjaTrader processes the orders lightning fast, but the broker API cannot handle the speed and starts to choke. If you want to test any frequency that’s not ultra high frequency, I recommend programming in NinjaTrader.

The FIX Protocol is the best option for the institutional trader that cares about maximal performance and does not suffer from the usual budget constraints. FIX is a fancy way of controlling communications between a custom platform and the broker. It does not involve software, only rules. The FIX protocol allows the trader to write software 100% from scratch. The trades and orders can go out the door literally as fast the machine can process them. It’s the advantage that comes with building everything from scratch.

Optimize an Expert Advisor

February 20, 2012 by Shaun Overton 1 Comment

One of the lesser known features of the MetaTrader backtester is the optimization feature. It’s so small that you could be forgiven for overlooking it.

Optimization is the process to maximize a certain outcome. In this case, it’s profit. Any EA developer wants to maximize the amount of profit made over a given period of time. The MetaTrader optimizer allows the trader to search for the combination of inputs that yielded the maximum profit over a given period of time.

The process is identical to running a backtest, except that MT4 runs multiple backtests at the same time. It then organizes the results and offers up the best combination.

Telling the backtester to run in optimization mode is easy. Simply put a check next to the word Optimization. MetaTrader will then sort through the combinations that you tell it to consider.

Place a check in the box next to Optimization in the MT4 backtester

The next step is to click on the Expert properties button to the right. A new window appears that contains three tabs: Testing, Inputs and Optimization. These screens allow the trader to inform MetaTrader which variables to consider for testing and how to weight the results.

Testing

The top of the testing section applies to every type of backtest. Here you can select the starting balance. MetaTrader defaults the option to $10,000, although you can make this any amount of your choosing.

The second default option allows the trader to restrict the direction of trades. It’s a frequent expert advisor programming request. It’s also one that is unnecessary. Both the backtester and expert advisor options screen allow the trader the option of restricting trades to long only or short only without additional programming. If the EA is not well programmed, this setting may cause errors 4110 or 4100 to appear all over the trading journal. It’s harmless. The only effect should be that the backtester slows down. It’s the result of writing to the journal hundreds of times or more.

The testing tab of the MetaTrader backtester

A groupbox appears underneath these options that inexplicably relates to the optimization process. You’d think it would make more sense to place it in its namesake tab. That’s typical MetaQuotes logic at work.

The first line contains numerous parameters for choosing the best option. User overwhelmingly select for the largest account balance, but other options include the profit factor, expected payoff, maximum drawdown and drawdown percent.

The last line automatically uses a genetic algorithm. Optimization processes use either brute force methods or genetic algorithms. Brute force strikes most people as intuitive although obviously exhausting. The software tests every combination possible. Genetic algorithm’s attempt to make the process more intelligent. When the software sees that certain parameters almost inevitably lead to a losing performance, the algorithm skips similar tests where it expects to lose.

This is a great idea if you have a quality genetic algorithm. My opinion of the MetaTrader backtester is less than stellar. I don’t feel very confident about the algorithm at all. If you don’t mind spending extra time waiting for test results then I suggest unchecking this option. You don’t want to miss a potentially important combination.

Inputs

Most people find this screen confusing. The first column, called value, strictly controls inputs for simple backtests. The Value column is totally ignored during an optimization run.

The inputs tab of the MT4 backtester expert settings

The important columns for this task are Start, Step and Stop. Start is the lowest number that the MT4 backtester will consider. Step refers to the interval between the lowest value and the highest value. Tightly controlling this setting allows the user to gain quick insights into how changing the variable values affects performance without dragging the tests out for a full week. Stop is the highest number that the expert advisor will use.

The most obvious candidate for testing in this example is the Take Profit value. The default setting is listed at 50. If you trade the majors, you might want to consider settings ranging between 10 pips and 200 pips. That means that you set Take Profit row to 10 for the Start column and 200 for the Stop column. The real trick here is selecting the Step. If you choose Step = 1, then MetaTrader will run a separate test for every value between 10 and 200. That’s 190 tests, which is overkill. A step of 10 cuts the total number of tests down to 19.

Optimization

This section is the nit-picky part. If a trader feels it’s unacceptable to have 10 consecutive losses in a row, he can place a check next the the Consecutive wins box. MT4 automatically discards any tests which yield a result that contains anything checked off.

The optimization tab in the MT4 backtester allows users to discard tests with undesirable traits.

When you finish going through each of the tabs, push OK in the bottom right corner. It’s time to launch the tests.

Curve fitting in the MT4 Optimizer

A word of warning: my personal opinion is that optimizing an expert advisor is usually a very bad idea. The unique settings that yield the most profit in 2012 are unlikely to yield the most profit in 2013. If you don’t control for random chance, there’s a good probability that the 2012 best combination may result in catastrophic losses in 2013.

I recommend that traders pursue any strategy development work in NinjaTrader. I don’t like the idea of optimizing at all. Instead, I always focus on testing strategies for entry and exit efficiency. I know from years of experience that these values never fundamentally change on instruments of the charts traded. Entry and exit efficiencies make wonderful metrics for automated trading because they are so stable.