range trading Archives - Algorithmic and Mechanical Forex Strategies

The guy that bet on Leicester City every year

September 5, 2016 by Shaun Overton Leave a Comment

Leicester City started the 2015 season with terrible odds of winning the Premier League Championship. Bookmakers only game them odds of 5,000:1 of winning.

To put that in context, you are more likely to die riding a bicycle than you were to win a bet on Leicester City. Or, you can think of betting on Leicester City every year. If you bet on them every single year for 5,000 years, you would expect them to win a grand total of… once.

2014 was hardly an indicator of their pending success. They were nearly relegated to a lower division (i.e., kicked out of the Premier League). And yet, they did win the championship last year.

Leicester City’s Biggest Fan

Meet John Michklethwait. He’s the former editor-in-cheif at The Economist and he’s currently editor-in-chief for Bloomberg. Clearly, he’s a very smart man. And yet, despite the odds and repeated disappointments, John bet on his old love, Leicester City, every single year dating back to the 1980s. That’s roughly 30 years of nonstop losing.

It wasn’t a lot of money each year: just £20. We all have our indulgences. I see the value of having skin in the game. £20 on a season is enough to make one care, but not so much that he’s upset about losing it.

Then something disruptive happened. John moved to the US last year for his position at Bloomberg. The chaos of the move threw him out of sorts, and he accidentally forgot to bet on Leicester City in 2015. He bet on them every single year dating back nearly 30 years. And yet the one year that he forgets to bet, not only did Leicester City win, but the bet paid out 5,000:1.

Let’s step back and calculate the cost of that oversight for Mr. Micklethwait.

£20 * 5,000 = £100,000.

A hundred… thousand… pounds. That kind of winning would put a nice dent in your mortgage, wouldn’t it?

The risk of low probability strategies

Everyone hears anecdotes about successful trend traders. Even winning only 30-40% of the time, they walk away big winners over time.

You live HERE. Math isn’t good enough. You also need to wonder if your strategy can handle real-world problems.

What if they took that even lower? They could move their stop losses closer to the market. They’d reduce the size of the average loser, but the winning percentage might also drop to 10-20%.

Mathematically, this could work out identically. 30% winners that earn 5x the average loser make for a profit factor of 1.5. A strategy with only 10% winners that make 15x the typical loser also have a 1.5 profit factor.

They’re the same. Aren’t they?

Planet Earth isn’t the same as planet Math. In the real world, people get sick and miss trades. Or, they move across the Atlantic and forget to place a £20 bet.

People move. They get sick. Computers break. Things can and will go wrong with trading.

Richard Dennis once commented that the Turtle Traders would often make their annual returns off of one, single trade. A single trade!

When your performance depends on positive outliers, you’re massively vulnerable to accidents. What happens if you’re sick that day? Or your internet goes down? Or your broker locks you out of your account on the worst possible day?

Life happens, brother. A plan that depends on perfection is no plan at all. You need to make yourself robust to failure. Or even better, you’d make yourself antifragile.

Winning percentages

I mentioned that you can do really well winning 30-40% of time. Why then, does my own trading strategy, Dominari, win 68% of the time?

Because I’m exploiting compound, exponential growth. It’s not just how much you win, but the order in which you win it.

Let’s take two examples:

A ranging strategy with a profit factor of 1.3 that wins 68% of the time.
A trending strategy with a profit factor of 1.3 that wins 30% of the time.

Look at the red circles. Trending strategies are prone to extreme outcomes, both positive and negative.

Each strategy risks about 1% on any given trade. And, the average of the range and trend strategies are identical in the long run.

But… and this is an important “but”, the expected worst case scenario with the trending strategy is substantially more likely compared to the range trading strategy. In effect, the average is more average with a ranging strategy than with a trending strategy.

Why is that? Because unusual losing streaks are devastating to trending strategies. Have you ever had a losing streak? It happens to everyone.

By using a strategy with a higher winning percentage, you’re making yourself robust to streaks of losers. And, not to mention, your average length of a winning streak is considerably higher.

Even though you’re getting the same mathematical outcome, you’re making things much easier on your trading psychology when you adopt a strategy with a higher winning percentage.

Dominari & Exponential Growth

You may have thought to yourself, “68%? That’s kind of a strange number to pick.”

You’d be right. The choice of 68% winners was not a coincidence. It is, in fact, the win rate on my Dominari strategy.

Dominari is about more than just buying and selling. Trading is also about managing a portfolio and position sizing. Position sizing is phenomenally important over your trading career.

My backtest results for Dominari show that for every $2,500, the account increased to $17,855.35 after 3 years. That kind of compound growth doesn’t happen by accident. That’s why I’d like to share the good news with you in my webinar this week.

I’m going to show you how to put that exponential awesomeness to work in your trading account. Sound good? Click here to register for the FREE webinar.

43 million real trades reveal the tactic of profitable forex traders

June 20, 2016 by Shaun Overton 4 Comments

Traders that follow one simple rule are 3.118 times more likely to be profitable 12 months later than those that don’t.

The critical feature of profitable traders is their reward to risk ratio. Yes, you’ve probably read that before, but this time it’s backed up with research. FXCM studied 43 million real trades from traders around the world to produce this analysis.

Image credit: DailyFX

Everyone “knows” that 90-95% of traders lose money. The good news is that the real percentage is noticeably lower. 83% of all traders lose money. And, that’s among the worst group. When traders use a reward to risk ratio of 1 or more, 50% of all traders are profitable after 12 months.

Be warned: the phrase “correlation is not causation” very much applies here. I cannot promise you that based on the data that using reward to risk ratios greater than 1 will automatically give you 50-50 odds of being profitable in the long run.

Logic, however, suggests that using good reward to risk ratios is a good idea. The advice to use reward-risk ratios above one appears in every trading book ever written for a good reason.

When traders use a reward to risk ratio of 1 or more, 50% of all traders are profitable after 12 months.

I suspect that it’s not the ratio itself that’s important. Instead, a large ratio discourages the worst mistakes that traders make.

I remember a project when I worked as a broker at FXCM. The systems desk analyzed the trades of the company’s most consistent losing traders. Perhaps taking the opposite signal of the worst traders might lead to profitable trades?

Alas, we found something far more mundane: the worst traders lose because they over-trade.

Trading costs

Think about how trading costs apply to the reward risk ratio. If you earn $2 for every $1 that you lose, it makes scalping an impossible activity

Traders using a 2:1 ratio need to use more patience. Even though FXCM offers low spreads and commissions, a 2:1 reward risk ratio implies further distances to the profit target. Longer pip distances lower the cost of every pip of profit.

Cost examples

FXCM averages a 1.4 pip spread on EURUSD. Let’s see how our reward-risk ratio affects trading costs using the 1.4 pip spread for our 2 examples.

Scalping

Profit target: 10 pips
Spread: 1.4 pips
Spread as a percentage of the profit target: 14%

Intraday Trend Trading

Profit target: 50 pips
Spread: 1.4 pips
Spread as a percentage of the profit target: 2.8%

Your cost as a percentage of profit in these examples are 5x higher when you scalp. That’s not good!

Holding trades with bigger profit targets minimizes the impact of trading costs. Said another way, you get to keep more pips when you win by increasing the distance of your profit target from your entry price.

The advice to use reward-risk ratios above one appears in every trading book ever written for a good reason.

Following a reward risk ratio greater than 1 naturally pushes you towards lower trading costs. Lowering your trading costs logically suggests you have a higher likelihood of long term profitability. If you want to get other critical tips for similar results, then make sure to sign up for the Foundations of Profitable Trading Checklist.

Reward risk ratio explained

The reward risk ratio compares your average profit to your average loss. If your average winning trade is $30 and your average losing trading is $15, then you have a reward risk ratio of 2:1. If your average winning trade is only $8, but your average losing trade is $16, then your reward risk ratio is 0.5:1.

Does the winning percentage matter?

Amazingly, the percentage of winning trades doesn’t seem to matter. The high frequency trading firm Virtu is a great example of this. Virtu wins on 99.999% of trading days even though it only wins on 49% of its trades.

The FXCM data shows that the average trader wins more than 50% of the time. EURUSD trades won 61% of the time, while some pairs were closer to 50%. The percentage of winning trades on all currency pairs is greater than 50%.

Image credit: DailyFX

Despite winning more than 50% of the time, trades with a poor reward risk ratio only had a 17% chance of earning a profit 12 months later.

… you get to keep more pips when you win by increasing the distance of your profit target from your entry price

If you’re currently struggling with your profitability, you’ve probably thought to yourself, “I need to win on more of my trades.” It’s like a business owner saying, “I need more customers.”

Smart business owners know that finding more customers is time consuming and expensive. It’s often much easier to sell more stuff to the customers that you already have.

It works the same way in trading. Instead of worrying about winning more often, you should focus your efforts on squeezing a few extra pips out of your winning trades.

If there’s anything that you should learn from this research, it’s this: the fastest way to improve is to earn more pips on your winning trades. You do not need more winning trades to do better.

Types of strategies with good reward risk ratios

The type of strategy that you select almost automatically dictates your reward risk ratio. Ranging strategies usually have ratios less than 1, which the FXCM data shows have a 17% likelihood of long term profitability. Trending strategies have ratios greater than 1, which have 50% probabilities of long term profitability.

Ranging strategies

If you daytrade EURUSD where the daily range has recently been around 80 pips, then that 80 pip range is the hard ceiling of what you could possibly make in a day. You know from experience that getting the bottom tick or the top tick of the day almost never happens. If you’re lucky, you may enter within 10-20 ticks from the bottom.

Upon entry, you also need to give the trade breathing room. That stop loss probably needs to be something like 25 pips if it’s a tight stop or 40 pips in order to have plenty of breathing room.

The best exits in a ranging market occur in the middle. You don’t know if the market will push back to its ceiling. It has just as much chance as going back to support and it does up to resistance.

The mid point of an 80 pip range is 40 pips, but you’re likely entering 10-20 pips from the true bottom. That only gives you a potential range of profit targets from 20-30 pips.

The most realistic, good ratio is a 30 pip profit target on a 25 pip stop loss, which is 1.2. Most strategies will probably risk 40 pips to make 20, which is a ratio of only 0.67.

Consider what a range trading strategy is. The market is stuck. It’s having a hard time going anywhere. You should only range trade if you have a well researched strategy with a long term edge. Otherwise, the typical trader is 83% likely to walk away with losses after a year.

Trending strategies

Trend trading strategies should last for weeks or months at a time. Looking again at EURUSD on a multi-month time frame, the current long term range is from 1.05 up to 1.16. That’s a range of 900 pips, but it’s not like the market wobbles up and down through that range. Instead, it gets stuck near 108, then briefly pushes down. It comes back to 1.08, then pushes up to 1.12. It might push up again to 1.15, then trade back down to 1.08. It’s hard to guess whether the next move will be up or down.

A 3,498.4 pip move in the EURUSD over a 10 month period.

Better long term plays are to sit on trades and let them pick a direction. The best recent EURUSD example began on May 8, 2014 at 1.39934 and ended March 13, 2015, at 1.04946. That’s a colossal 3,498.4 pip move in just 10 months.

Is there a scenario where you’ll risk almost 4,000 pips on a trade? Of course not. What about 1,000? No! What about 500? No!

The natural risk reward ratio for these types of trends is astronomically high. For a few hundred pips of risk, you can make 10 or more pips for every one risked.

As long as you’re not aggressively trading, trending strategies are far more difficult to mess up. If you can click a button, enter a stop loss and then do nothing for months at a time, then you’re qualified to consider trend trading.

The practical application is of course more difficult than that description, but that’s the idea in a nutshell. If you’re a newbie forex trader and wondering where to start, long term trends are the place where you’re less likely to get hurt.

The problem for newbies, though, is that they’re looking for excitement. It’s not terribly exciting to place on trade and then do nothing for months. It’s one of the paradoxes of the market that less work can often lead to better results.

How to improve your trading

The reward to risk ratio is a critical element for new traders to increase their chances of success, but it’s not the only one. Click here to register for our free Foundations of Profitable Trading Checklist. You’ll learn simple, but useful, tips to improve your trading.

The Ultimate Signal for Range Bound Trading

August 19, 2015 by Lior Alkalay 6 Comments

How does one identify that a range bound opportunity is descending upon us? We all know the trading signals for identifying trends and trend breaks, but how about those that identify a range bound? While most of us focus on riding the next trend that may not be where the next opportunity lies.

That might be in a range bound, where it’s easy to figure where the pair will move lower or higher. Of course, we all know what a range bound looks like in the charts. The problem is once we identify it as such, it might be too late. Many of the range bound trends become apparent only in retrospect, when the pattern crystallizes. But there is some good news. Just like tools that identify trends and trend breaks, there are effective tools for identifying the approach of a range bound.

Range Bound and Moving Standard Deviation

The single most evident quality of range bound is, quite simply, falling volatility or Standard Deviation. As Standard Deviation falls, the pair has smaller fluctuations and therefore is “bound” within a range. Likewise, when Standard Deviation is low we are stuck in a range.

If we can time Standard Deviation (and we can), we can know when it’s about to fall. If it is about to fall then we are heading to a range bound and can adjust our strategy accordingly. Moving Standard Deviation is a measure of the change of Standard Deviation through time. And it is exactly designed for our specific case and thus makes it the ultimate tool to time an upcoming range bound. Here’s how you do that:

Source: e-signal

In the chart above, we have the Moving Standard Deviation or MSD running (at the bottom). It is evident that MSD tends to top out after surpassing the 80% level while it tends to bounce back after falling below 20%.

In part A and C, you can clearly see that as MSD topped out above 80% the pair became less volatile and then moved into a range. On the flip side, when MSD moved below 20%, we can expect the range bound to end and the trend to continue.

Validate the Range

First Validation: If the pair had been surging when the MSD surpassed the 80 level, that’s your high band of the range. On the other end, if the pair had been sliding when the 80 was reached, that’s your low. Whatever high or low was reached during the time the MSD hit above 80 that’s your first confirmation of the range you will be getting.

Second Validation: The confirmation of the other band of the range will present itself as a candle with a needle, rather shortly after.

A Few Rules of Thumb

MSD 20 – Exit Immediately: Regardless of direction in your range bound trade (long/short) or time elapsed since the 80 level signaled the range bound, be wary. If the MSD falls below 20 get ready to terminate your trade, because the trend is about to resume and your range might be over. Although it’s not always immediate, it’s better to take precaution rather than lose your gains by ignoring the below 20 signal.

MSD is Only Good for Range: That might sound tricky. As we just stated, the MSD will always fall below 20 before the trend resumes. You’ll know when to exit your range bound trade and be able to cap your risk of loss. But many times MSD will fall below 20 and the trend won’t immediately resume. Thus, don’t use the MSD to predict the opposite of a range bound; that is, to predict the trend. There are different tools for identifying when a trend is about to re-emerge.

Range Bound Cycles: Finally, as you can see in the chart, in the long run the cycles of range bound vs trending tend to be more or less the same length. That is always something to keep in mind. Although the MSD will be your mark to exit/enter the range, timing your cycles of range is always useful to let you know how much more time might be left in that lucrative range bound trend you are taking.

Tactics for Managing a Trade in a Range

March 11, 2015 by Richard Krivo 5 Comments

Range trading is simply finding a currency pair that has been moving between a defined level of support and a defined level of resistance for an extended period of time.

Once the setup has been identified, the trading plan is to buy at support and sell at resistance as long as the range remains intact.

Let’s take a look at the range on the historical 1 hour chart of the NZDCAD below…

In this straightforward scenario when price trades at support the trader would take a long position with a stop just below the lowest wick in the range. The limit (take profit) would be set at the top of the range and the trader would let the trade play out. If all goes according to plan and price trades up to the limit, a profit of 40 pips would be gained. If price retraces and take out the stop the trader would incur a loss of about 8-10 pips.

Now let’s take a look at a few trade management variations on this same trade…

Here we have the same trade set up but we are going to increase our likelihood of taking profit by moving our limit down slightly – a bit closer to our entry.

Here’s why this can work. Notice that price has hit the very top of the range four times. (This would be each time a candle body or wick touched or pierced the .8130 level.) On the other hand, price has touched our pierced our new take profit level a total of 15 times!

By doing this we will give up some profit if the pair does trade to the top of the range. However, we are increasing our odds of having our limit hit while still achieving a solid profit and having at least a 1:2 Risk Reward Ratio in place.

The choice is yours…

Take the chance of gaining more pips but with a lower probability of success or take the chance of gaining a few less pips but with a greater probability of success.

Now let’s take a look at the same set up but this time we will open two positions instead of one…

Personally, I like to trade multiple lots since it allows me more flexibility in my trade management.

Keep in mind, however, that anytime we increase our position size, we also increase our risk.

For example, if our stop is set at 10 pips, with one lot we are risking 10 pips; with with two lots we are risking 20 pips and so forth. To stay within our Money Management guidelines of never placing more than 5% of our trading account at risk, check to be sure that the trading account can handle the added risk.

When looking at the chart above, the set up remains the same but this time we would open our trade with two lots instead of one. (This strategy would remain the same whether we open two, six or ten lots.)

The flexibility that this allows is that we can close out one or a portion of the trade when price reaches the halfway point in the range. By doing that the trader locks in profit when that position is closed. Additionally, the stop at that point can be moved to breakeven , that price at which the trade was entered.

Now, if price retraces and hits our stop we are closed out of that portion of the position with no gain but no loss is incurred either. We also have the 20 pips from the first portion of the position locked in.

Should price continue to move to the top of the range, the second position would be closed out thereby gaining the full amount of pips encompassed in the range. In the case of this example that would be 40 pips. Those pips would then be added to the profit gained when the first position was closed out at +20 pips for a net gain of 60 pips.

Bottom Line: There are a variety of ways to manage the same trade setup. Choose the one that makes the most sense for you and the size of your trading account.

All the best and good trading,

Richard Krivo

RKrivoFX@gmail.com

#RKrivoFX

Walk Forward Optimization

January 13, 2014 by Shaun Overton 6 Comments

If you were walking and randomly it started to rain, would you consider carrying an umbrella tomorrow? Of course you would.

The reason I ask a rhetorical question like that is when people observe a behavior, they respond accordingly. If they expect that something might happen again, they change their behavior to accommodate the change in outcomes.

When you think about forex robots, everybody has the dream of developing a strategy that works forever. It requires no changes. The initial settings always work. Turn it on and move to the beach.

Reality, of course, is more complicated than that.

Walk forward optimization continually optimizes throughout time instead of looking for one set of static settings

That leads to expectations of what you need to do when your strategy inevitably goes awry. It’s very possible that you come up with a strategy that works and does amazingly well on the current market. However, a past genius doesn’t mean future genius. There’s always the chance that your strategy will no longer work in the future.

Why is that? It’s the same reason that you might carry an umbrella tomorrow if it rains today. People observe the market performing in a consistent manner. As more and more people make the observation, people start trading on it. The market responds to those changes, and eventually the opportunity completely washes out as too many people have eared about it.

Walk forward testing is the process of determining whether or not your strategy has washed out. By testing on one set of data, and then testing it on a blind set, you can give yourself an indication of whether your strategy is bad or not. The goal of walk forward isn’t to prove that your strategy is good. It’s to prove that your strategy is not known to be bad.

The process of walk forward testing is very simple. You identify a set of information that you want to use for your testing and optimization. Using a real example, right now it’s the beginning of 2014. So maybe you want to look and test data from 2011 through 2012. That would be your in sample data, and then your out of sample data might be all of 2013.

In order to conduct a walk forward test, you would test and analyze your strategy 2011-2012. Then, to determine if it’s “not known to be bad”, you then walk forward to 2103 to see review the performance.

What you’ve done is a blind test. You didn’t know what how the strategy would perform in 2013 when you tested it in 2011-2012. By putting it on a blind sample, you give it the opportunity to fail.

The reason so many traders put their faith in walk forward testing is because it’s the absolute best tool to identify weaknesses in your optimization. When you’re testing a strategy, it is very likely that you’ve overfit to past opportunities.

Self feedback loops in the current market

Let me give you an example. In the current markets, a lot of traders have been banging gold on the market open where every day at market open., they sell as much gold as they possibly can. Sometimes it’s several multiples of the annual production in a span of a few minutes. What you see is an absolute freefall for five or ten minutes. That state persists for days at a time. But that doesn’t last forever. When enough traders start seeing that people bang gold on the open, they start doing the same thing.

Effectively, whoever wants gold to falloff on the market open has taught other traders to do that trade for them. As people expect gold to fall in the first five minutes of the open, they then change their behavior. Some try to jump on banging the open and go short.

Others start modifying their behavior. They notice that gold free falls for five minutes. Then, all of a sudden it stops, and more than like it reverts to the mean. They’ll start changing their tack and buying after so many minutes have elapsed from the open. They expect that the heavy volume that preceded the selling will eventually return to normal. As people change their behavior, other people respond in kind.

If enough people start selling on the open and then buying on the open five minutes later, you can see that a pattern is forming where one person responds to the actions of another. It’s a self feedback loop where the state that was working for the first couple of days no longer works in the future.

If you can identify a strategy that is able to survive those conditions, and is able to survive conditions where you didn’t do any testing and optimization, you give yourself better odds of succeeding in the future. It means that not very many traders have clued into this trading opportunity that you’ve discovered.

The approach to to walk forward testing is the antidote to the problem known as curve fitting. Curve fitting is the ultimate woulda coulda shoulda strategy. It’s akin to opening a chart from yesterday and saying I would’ve bought here and I would’ve sold here, already knowing what transpired.

Of course you’re going to “make money” in that situation. You know with perfect information what the market did. In the future, you don’t know the perfect information. The goal of a strategy is to deal with that ambiguity.

Curve fitting means that you’ve fit everything so perfectly to past market conditions that when new situations inevitably arise, sort of akin to the phrase, “history doesn’t repeat itself, but it rhymes,” your strategy does the same thing.

You want a strategy that does well on past performance, but you’re not coming up with a strategy to make money on historical markets. The purpose of developing a strategy is to make money in future markets. When you’re backtesting, you’re trying to strike the balance between solid historical performance and, most importantly, making sure that that historical knowledge extrapolates to future performance. Your goal is to make money.

Rolling Walk Forward Optimization

Rolling walk forward optimization takes the walk forward idea and continuously improves the strategy by exposing it to new data. So let’s say that you have a twenty four month sample period. One way to go about it would be to optimize your strategy for a period of two months, then to walk it forward to the third month. You observe the behavior and you reoptimize for the second and third month, then walk it forward to the fourth month.

By doing so continuously, you eliminate the decay time of the strategy and give it a chance to adapt to ongoing market conditions. It is sort of the redheaded stepchild to machine learning. Experience and losses give the strategy the opportunity to improve and adjust to the market changes through walk forward optimization.

…you eliminate the decay time of the strategy and give it a chance to adapt to ongoing market conditions

Another important consideration for walk forward analysis is the degrees of freedom within a system. For example, let’s say that you are analyzing a moving averaage cross. You’re using two moving averages and use a fixed stoploss and take profit. That would give you four degrees freedom. The fast moving average is the first degree. The slow moving average is the second degree. The third is the stoploss and the fourth is the take profit.

The more degrees of freedom that you allow in a system vastly increases the chances 0f curve fitting your systems to historical data. The absolute best systems maintain twelve degrees of freedom or less. You want to find trading opportunities that have large numbers of trades and that offer performance that you find satisfactory.

Another element to consider in your optimization is what are you optimizing for. Most people focus on the absolute return. Returns are great, but most traders care much more about how they make their money instead of how much. Let me give you an example. If I had a system that made $25,000 last year, would you want it? Almost everybody says yes.

If I have a system that made $25,000 last year, but you had to lose to $15,000 before you made any money. Most people don’t want that system. What this means is that you care a lot more about the performance on a day-to-day basis rather than end result. The problem with optimization and even walk forward optimization is that you’re not necessarily focused on what you care about in the real world: the way that you’re making your money.

Most charting packages focused on the net outcome and that can cause some weaknesses in your system. If you’re range trading, what you’ve really done is cherry pick the results that are the least affected by substantial news. In effect, you’ve chosen the settings that have not yet been affected by fat tails.

If you’re trend trading, you’ve done the exact opposite. You intentionally pick the settings that maximize the fat tailes that have happened in the past. With trend trading strategies, you probably aren’t going to find consistent performance. Instead, what you’ll find is that the optimization frequently causes long, ongoing droughts of incessant drawdown. Then suddenly, almost out of nowhere, it finds a mega monster winner that returns several multiples of the drawdown that you experienced. This is fine for a hypothetical backtests, but in the real world where you’re suffering losses on a near daily basis, most traders can’t take the pain. The weakness I find with most optimizations is that they don’t look at the consistency of performance. A potential substitute for optimizing a strategy would be looking at the linear regression of the equity curve over time. The best equity curve has the strongest linear regression slope.

Popular charting packages that implement rolling walk forward optimization are Amibroker, TradeStation, Multicharts and NinjaTrader.

Walk forward optimization in NinjaTrader

Open the Strategy Analyzer from the Control Center. Click File / New / Strategy Analyzer.

Open the strategy analyzer in NinjaTrader

Left mouse click on an instrument or instrument list and right mouse click to bring up the right mouse click menu. Select the menu item Walk Forward. You can also click on the “w” icon in the Strategy Analyzer toolbar. If you prefer hot keys, you can also use CTRL + W. Lastly, you can also push the “W” icon at the top left of the Strategy Analyzer.
Select a strategy from the Strategy slide out menu
Set the Walk Forward properties (See the “Understanding Walk Forward properties” section below for property definitions) and press the OK button.

There are many ways to select walk forward optimization in NinjaTrader

The Walk Forward progress will be shown in the Status Bar of the Control Center.

Ranging SMA 200 Walk Forward Results

February 25, 2013 by Shaun Overton 4 Comments

We have come a long way. The original group trading strategy anticipated the need to resort to a complicated money management strategy. A simple range trading strategy emerged that made complicating steps unnecessary.

Testing from 2011 showed the best chance of earning a profit came from the following settings on the EURUSD:

30 minute chart
Price exceeds 1.5% or more of the 200 SMA
Buy/sell at market on an expectation of reverting to the SMA

The tests showed a hypothetical net profit of $1,310 trading 1 standard lot per signal. Assuming that you’re comfortable using 10:1 leverage, that makes for an annual return of 13.1%. Higher leverage increases the return at increased risk. Lower leverage decreases the return at decreased risk.

Walk forward results

The walk forward results are profitable! That is a huge relief, especially given the amount of effort put into the research.

The thing that disappointed me the most, however, is the huge drop in the number of trades. The original strategy placed 60 trades in a 12 month period. The walk forward test only traded 22 times.

The equity curve shows a substantial time lag in between trades. The blind test shows a profit.

The equity curve makes it obvious that nothing happens for large periods of time. Over six months pass between the first trade on January 19, 2012, and the next trade on June 29.

Thereafter, the pace really picks up. September 2012 showed the most activity with 12 total trades – more than half of the year’s activity happened in that individual month.

The gross outcome is a profit of $580. The cost of trading, assuming a 2 pip spread, is $440. The final return is $140, or a 1.4% return on 10:1 leverage.

Metrics

The results for the walk forward tests, as shown in NinjaTrader

Trading efficiencies of the 2012 walk forward results

The entry efficiency

The exit efficiency

Analysis

The substantial drop in the number of trades relates to something that we already knew. The trading strategy needs volatility in order to find trading opportunities.

Whenever volatility drops, which forex brokers bemoaned at the Forex Magnates conference in London, the number of trading opportunities drops, too.

The walk forward test indicates that this is a great strategy to keep in my pocket for when volatility picks up later this year. It held up on blind data. Although the higher profit facotr likely results from the small sample size, it feels reassuring when the metrics improve on the blind data.

I allowed myself to cheat slightly where I tested the 1% setting in 2012 instead of the 1.5%. The performance showed a similar drop in the number of trades and in the profit. Importantly, however, the numbers in that sample set improved.

I like this strategy because it is stable. Changing the setting from 1.3% to 1.5% does not cause a drastic U-turn in the performance. In other words, it’s steady and predictable.

I would feel far less confident in the outcomes if minor changes in the settings yielded enormously different profits or losses. Violent shifts in outcomes from small changes relate to chaos theory. That’s not a desirable trait in an algorithmic system.

Future improvements to the strategy might relate volatility to the entry setting. When volatility is low, the 1.5% threshold might lower to 1%. As volatility rises, a band of 2% might yield better outcomes in extreme scenarios. An obvious next step would be to relate the movement of the band to the volatility of the EURUSD itself.

After-thoughts

This series eventually led to a profitable trading strategy. If you’d like to read through the journey, then I suggest reading the articles sequentially

The initial strategy idea
Selecting an appropriate time frame
A research plan
An annoying surprise in the initial backtests
An attempt at range trading
Range trading results
The moving average envelope scalper

Range Trade at High Frequency

February 28, 2012 by Shaun Overton Leave a Comment

Range trading systems make the best candidates for high frequency systems. They are less execution sensitive than trending systems for a simple reason. Range trades “catch the falling knife,” making them suitable for using limit orders.

High frequency prices vary from the normal M30 and H1 charts. The lower the time frame, the better that the chart fits to a normal bell curve. One common theme in systems trading since the 2008 crash has been “tail risk” or “fat tails”, which refer to the edges of a probability distribution like the bell curve. The fatter the tails, the more likely that a range trading system is to crash and burn.

The high frequency bell curve shows the tail risk of important events

The bell curve shows the tail risk of important events. The tails are colored in red. Fat tails mean that important news happens more frequently

The real world events captured in the tails reflect headline news like Bernanke speaking or Ireland announcing another referendum on all this bailout nonsense. The events only happen once, obviously. If you consider the news events in the context of hourly charts, they happen frequently as a percentage of the overall period. If you look at a one minute chart, that same event is now about 1/60th as important. Dropping down to tick charts nearly makes the events disappear in the statistical profile.

My experience is that the news cycle drives trends on a macro basis. “Macro basis” and high frequency are two topics that don’t belong together. Trending systems should focus on long term trading, while ranging systems are far more suited to high frequency. If your system trend trades, you can throw it in the rubbish bin for high frequency trading ideas.

High frequency considerations

Keep in mind that there are effectively two ways to participate in the forex market: you can either act as a price taker or as a price marker. Price takers range across all market participants. A hedge fund or university endowment is just as likely to take a price as they are to make one. CTAs and retail forex traders are much more likely to make their decisions based on the expected market direction. Timing is critical for them, so they don’t want to leave it to chance whether or not they’ll get to enter a trade.

The trader gets filled right away. That’s the major advantage. The main disadvantage to acting as a price taker is that you pay the spread every single time that you want to enter a position.

I sat with AvaFX in Dublin on my last trip. They charge a 3 pip fixed cost spread. I mentioned my concern about how that spread affects my client’s EA performance. His MetaTrader expert advisor trades 4 times per day on 2 currency pairs. If you do the math on a 3 pip spread, it works out to 8 * 260 = 2,080 trades per year. If you’re paying 3 pips and trading a $10,00 account, you would have to earn $6,240 per year – a 62.4% return, just to cover trading costs. I don’t care how good a system is – it will never cover those kinds of costs. Trading on margin will not do anything to resolve the issue. Spread costs are directly proportional to the amount traded, which impacts the profit. There is no way to trade and make money if the transaction costs are too high.

Designing an expert advisor is difficult enough, but it’s even harder when you factor in the trading costs. Say, for example, that I develop a EA that wins 75% of the time with a payout of 0.5:1 before trading costs. When the EA wins, it earns $0.5. It loses $1 whenever a loss occurs. The profit is 75 wins * $0.5 = $37.5. The loss is 25 * $1 = $25. The expert advisor’s profit factor is 37.5/25 = 1.5.

That should sound great. The problem occurs when the total commission outweighs the total expected profit. This example required 100 trades. Let’s say that we were trading mini lots with an average win of 5 pips and the average loss of 10 pips. That puts the gross profit at $375 and the gross loss at $250. The return is $125 for the 100 trades, excpet that we must now subtract the $100 for trading costs. The total profit plummets to a measly $25.

If the expert advisor’s expectations held true for something like a 10 pip take profit and 20 pip stop loss, the trader might be better off to change the exit points. The reason is that the profitability may actually improve. The goal would be to reduce the number of trading opportunities with an eye towards making them more profitable relative to the costs.

A better approach, in my opinion, would be to switch over to market making. Although you usually still pay to trade, the advantage to market making is that you earn the spread rather than paying it. The spread is overwhelmingly most traders biggest cost. Not paying it opens the possibility of applying the strategy where one normally could not afford it.

Market making only works if your forex broker allows you to post best bid/best offer and have the price reflected on the screen. Most brokers claim that they are ECNs. A real forex ECN allows you to post limit orders. Whenever that order represents the best bid or offer, the price and size of your order shows up on the screen. The only retail trader friendly brokers that I know of are Interactive Brokers and MB Trading.

I ran my NinjaTrader license at MB Trading last week to test the execution and order flow. The test only use traded a microlot (0.01) and posted best bid or best offer on the EURUSD. The orders remained valid for anywhere from 1-10 minutes. Despite the small trade size and lengthy time period as best bid/offer, the orders only filled 75% of the time. That meant that I caught 100% of the losers but only 56% of the potential winners. Not good, in spite of getting paid for the limit orders.

Interactive Brokers is the next test candidate. They have been around much longer and should have far more order flow. I’m hoping that the low fill rate that I experienced making a market at MB Trading will improve substantially when I shift the same strategy to Interactive Brokers.

I expect to find a few other changes as well. The spread that I earn should fall from around 0.9 pips on EURUSD to 0.5 pips, which is indicative of Interactive Brokers’ improved pricing. I also will have to pay a 0.2 pip commission, which reduces the net credit from 1.0 pips at MB Trading (0.9 spread + 0.1 commission) to 0.3. Nonetheless, I expect the improved fill rate on winning trades to work more in my favor.

The thing that most people will hate is that you can only test a market making approach with live money. It’s sufficient to backtest a strategy using market orders with a 0 spread assumption. The goal is to weed out the junk from diamonds in the rough. No method exists, however, to accurately determine whether or not a trade would have gotten filled with a limit order. The only way to find out is to test an idea with live money, then to compare the results to a backtest over the same period. If the live, high frequency performance is similar to a backtest, then you probably have a winning approach.

The real motivation here is to get as many opportunities as possible. Just like the casino does everything to help you pull the slot machine faster, the trader should look for as many favorable setups as possible. High frequency stands out in this area. The inherent advantages of a system are more likely to manifest more quickly. Assuming that you get a handle on the trading cost problem, the profit is often limited only by the number of trades that can be squeezed into a day.

Programming options at high frequency

MetaTrader 4 is not a good candidate unless you expect to post orders once per minute or slower. MetaTrader suffers from the Trade Context is Busy error. Running an expert advisor on more than a single instrument could cause orders to enter too slowly or not at all. MetaTrader is only an option with MB Trading. Interactive Brokers does not support MetaTrader.

NinjaTrader works great and offers a lot of the broker portability that comes with programming in MQL. Programming a high frequency strategy in NinjaTrader works at most human speeds (5 seconds or more). For the brokerages where NinjaTrader submits orders using the broker’s API, I find a speed bump affect at work. NinjaTrader processes the orders lightning fast, but the broker API cannot handle the speed and starts to choke. If you want to test any frequency that’s not ultra high frequency, I recommend programming in NinjaTrader.

The FIX Protocol is the best option for the institutional trader that cares about maximal performance and does not suffer from the usual budget constraints. FIX is a fancy way of controlling communications between a custom platform and the broker. It does not involve software, only rules. The FIX protocol allows the trader to write software 100% from scratch. The trades and orders can go out the door literally as fast the machine can process them. It’s the advantage that comes with building everything from scratch.