NinjaTrader Tips

Trading Platform Limitations

October 18, 2015 by Shaun Overton 6 Comments

This post was authored by Ben Fulloon, a respected trader and subscriber to OneStepRemoved.

I developed an awesome strategy with a drawdown ratio of 13.67. Sounds amazing, right? Too bad that my trading platform overstated the results by more than double!

It’s important to learn about both your brokers and platforms limitations. Sometimes these intricacies only become apparent through time and experience. It’s so frustrating when your trading platform doesn’t function or report results as expected.

In this article I’ll point out two limitations of NinjaTrader 7, one bad limitation and one which can actually turn out surprisingly better for the trader in certain situations. However, this is more to do with the broker I’m using and not the platform itself.

NinjaTrader is definitely not the only platform that has limitations: MetaTrader, TradeStation, X-Trader, Matlab, etc. all have limitations for quantitative finance.

I’ll just be writing about NinjaTrader in this article to keep it fairly short and easy to read. I am also not intending to make out NinjaTrader as being a bad platform either. But, there are definitely some improvements that could be made to make it a lot easier and more convenient for quantitative traders to develop and trade strategies.

The first quirk relates to the broker I’m using. Specifically, it’s the day trade margins that I care about. These day trade margins end 15 minutes before the close of the session. For instance the ES (Emini S&P500) has a day trade margin of $500, which ends at 4:00pm CT that then reverts back to the full trading margin of $5060 before the session closes at 4:15pm CT. (Times stated are correct at time of Writing, The ES now closes at 4:00pm CT and the Day Trade margin ends at 3:45pm CT)

I’ll show you a screenshot of the results of a day trading strategy that I developed. This strategy trades the ES, NQ (Emini Nasdaq 100) and the YM (Emini Dow) all at the same time. The easiest way to exit on close with NinjaTrader is setting “Exit on Close” to true which will then exit on the close of the session.

According to the results the strategy makes a total of $332,771.60 with a maximum drawdown of $25,912.27 since 2008 to now. This is a drawdown ratio of 12.84. That’s oustanding!

The issue is… and you knew there’d be a problem… is that the strategy exits at 4:15pm CT. Day trading margin ends at 4:00pm CT. The strategy is therefore highly likely to get a margin call with a small account size.

It makes sense to tweak the strategy to make best use of the day trading margin. Ninjatrader offers a custom session template, which in this case I made end at 4:00pm CT. The results of the custom session template is as follows.

The exact same strategy applied to the same instruments to avoid a margin call makes $335,819.30 with a maximum drawdown of $24,560.51. This is a drawdown ratio of 13.67.

I didn’t change the strategy with the goal of improving the drawdown ratio AND the profit. But hey, I’ll take it. Finding a limitation in the platform can actually benefit you in some situations.

This strategy is based on trading 3 different instruments. The ES, the NQ and the YM. The problem is that I backtested it using an instrument list in NinjaTrader. What this means is they’re all tested separately. NinjaTrader then combines the test results for you as a total result like the results of the screenshots above.

Here’s what it looks like when you test them as an instrument list. This shows the different profits and drawdowns of the individual instruments.

Now at first glance it reads that the trader would have made $335,819.30 with a maximum drawdown of $24,560.51 if they traded all three instruments together. Don’t you agree?

The problem is that this is incorrect. NinjaTrader doesn’t actually combine the results like you’d think. The trader still would have made roughly that money. However, all the statistics aren’t quite correct.

To show this I recreated the exact same strategy however it will trade the ES, NQ and YM all at the same time instead of trading them separately like it does by default. These are the results when you program it into a multi-instrument strategy

It makes $335,915.30 which is roughly the same amount, but it has a maximum drawdown of $59,937.60 instead of the $24,560.51 it originally looked like it would be. This makes it a drawdown ratio of 5.60, which is a lot worse than the original 13.67.

If the trader decided to trade based upon the maximum drawdown of $24,560.51, they may get a nasty shock when the drawdown turns out to be twice as bad as they were expecting.

Incorrect calculations on such an important metric could jeopardize an account. You might assume that you can get away with half of the equity that’s actually required to trade the strategy. Oops?!?

The misleading statistics in NinjaTrader makes this strategy look really nice. But when the drawdown is more than double what it appeared that it would have been originally, you might get a nasty shock.

This is why it’s important to learn both your platforms and brokers limitations as early as possible. You don’t want to learn these limitations the hard way.

In a few weeks time, I’ll reveal a simple way to create multi-instrument strategies which show more accurate metrics. Stay tuned for my next article in the series.

Import Stock Symbols to NinjaTrader

May 5, 2014 by Shaun Overton Leave a Comment

Importing numerous stock symbols into NinjaTrader is a tedious process. If you’ve ever used the Instrument Manager and want to create a list of the most liquid ETFs, you know what I’m talking about.

There is a far, far easier way to import multiple stock symbols into NinjaTrader without using the Instrument Manager.

Go to the control center, the main part of the NinjaTrader Program. Click on File, Utilities then Import Stock Symbol List.

The new screen brings up a text area. Type in the list of symbols or click on the small Load button in the middle of the screen towards the left side. That allows you to import a list of tickers if they’re already typed into a file.

“Traded on” is the exchange where these instruments trade. You can use Nyse if you’re not planning to trade live. The symbol mapping is only important when it’s time to use NinjaTrader for live order execution.

My final tip is that you can assign all of these instruments to a list. The screenshot depicts a few commodity ETFs. It makes a lot of sense to categorize those instruments into an ETF list. You could click the “New” button to the right or, if you already created a list with that name, select it from the drop down menu.

A New Look At Adaptive Asset Allocation

March 13, 2014 by Eddie Flower 6 Comments

Adaptive Asset Allocation (AAA) was born as one of several sibling strategies for applying Modern Portfolio Theory (MPT), which was first proposed in 1967 as a way to optimize portfolio gains. Yet, many traders and financial strategists who truly believe in the math of MPT are disillusioned because the real-world results while using AAA haven’t met their calculated expectations for gains, and the volatility of such portfolios has been higher than expected.

Recent studies of this topic have suggested that this mismatch between expectations and reality may be primarily due to the length of the time periods used for input averages and portfolio rebalancing: Apparently, when calculations are based on input data using averages obtained over much shorter periods of time, the portfolio returns are better than when those averages are calculated based on long-term numbers. And, when the portfolio rebalancing intervals are shorter, performance is better and volatility and risk are reduced.

To recap, MPT relies on 3 parameters to create ideal portfolios, typically involving a set of asset classes including stocks in the U.S., European, Japanese and emerging markets, plus U.S. and international REITs, U.S. long-term and intermediate Treasuries, as well as gold and other commodities. The parameters are:

Expected volatility
Expected returns
Expected correlation

It seems that using shorter-term averages for MPT scenarios leads to more accurate results. One shortcoming of the previous-generation allocation model, Strategic Asset Allocation (SAA), becomes apparent because that model applies MPT based on long-term averages regarding the above parameters. As detailed in the recent new work on this topic, using long-term averages leads to significant errors in calculated returns.

In practice, long-term averages over a 5-to-20-year time horizon are poor predictors of volatility, returns and correlation. The statistical gap between calculations using 20-year averages and those using 3-or-4-year averages with regard to stocks’ annualized returns is huge, ranging from negative returns to nearly 14%. Given the relatively short investment time horizons of most investors nowadays, it seems clear that using shorter-term parameters in the calculations will yield more realistic results.

Portfolios offer better risk adjusted returns when they adapt to short term market conditions.

To acknowledge reality without disavowing longer-term calculations entirely, some investors choose to tweak their calculations by applying a long-term value approach instead of a long-term average approach, which tends to weight portfolios in favor of equities when stock prices fall, and conversely to reduce weighting in equities as their prices become more expensive.

Yet, with advancing technology there are some new alternatives to using long-term valuation for “handicapping” the calculated returns. At the extreme end of the short-term horizon lie the high frequency traders, who take advantage of short-term trends, correlations and reversions-to-mean in order to generate more-realistic estimates of returns. There is currently much excitement in the trading community based on the success of traders who use HFT systems. Still, as more traders crowd into this niche, it’s possible that the spreads will thin or perhaps vanish altogether.

The predictive value of momentum

Momentum is an excellent way for investors to estimate performance over the short term. According to the old adage: The best predictor of short-term future price is the current price. And, as the investment horizon is extended from intraday or daily trading outward toward weekly periods, the effect of momentum becomes more noticeable. Perhaps due to larger, slower-moving investors, prices tend to keep moving in the same direction for several weeks. Given this probability, it’s logical to account for momentum when building a portfolio, regardless of the long-term averages already observed.

Volatility

Volatility, too, has been misapplied with regard to MPT. For example, although average long-term annualized volatility is about 20% for stock prices and about 7% for 10-year Treasuries, actual volatility measured during the shorter time horizons of most investors fluctuates much more wildly, and is therefore much less accurate for projecting future conditions. So, actual volatility can have a far more adverse impact on a portfolio than the calculated volatility implies.

And, although many investors attempt to roughly balance the difference in volatility between stocks and bonds by weighting portfolios with 60% stocks and 40% bonds, still, the actual volatilities experienced can far override such a crude balancing method. Therefore, with regard to volatility assumptions it seems safest to rely on the adage mentioned above, that is, the least-biased guess of tomorrow’s price is based on today’s price. Likewise, the least-biased guess of tomorrow’s price range is the price range during the recent past, which of course represents the recent volatility.

Since recent volatility seems to offer the best guess about near-term future volatility, and most investors have a short-term horizon, it seems logical to use short-term volatility as the parameter for MPT instead of long-term volatility. As a takeaway regarding volatility, a savvy investor rebalancing a portfolio can calculate its volatility and, in order to maintain the volatility risk at a stable level over time, could reduce exposure by partly moving into cash when volatility exceeds the targeted level.

Correlation & returns

Even though long-term correlations between the prices of asset classes such as stocks and Treasuries, or stocks and gold, are low or negative, over shorter time periods the actual correlations vary greatly. So, for example, the volatility of a 50-50 stock-and-bond portfolio may decrease by 50% as the correlation decreases.

Similarly, although many traders intuitively understand that a portfolio’s risk is reduced by apportioning the volatility of its components, a less-intuitive observation from the recent studies has been that returns from risk-managed portfolios were also improved by as much as 25%. Finally, since the human nature of investors makes it difficult to focus on returns alone while disregarding risks, especially over a longer term when drawdowns may accrue, it’s also prudent to consider maximum drawdown along with volatility when seeking maximum returns.

Summary

If MPT scenarios based on near-term average values give more accurate estimates than those based on long-term values, then it seems best for HFT traders and other short-horizon investors to use current observed values for portfolio optimization. In the recent studies cited herein, the authors have advocated the monthly rebalancing of portfolios by using a true Adaptive Asset Allocation based on returns in the near term in view of their momentum, along with the appropriate short-term volatility and correlation averages.

One algorithmic approach might be to create fresh portfolios at the time of monthly rebalancing based on the top few assets according to six-month or even shorter momentum, and to allocate assets according to an algorithm specifying minimal variance in volatility, instead of apportioning each asset according to its individual volatility. This approach would account for the volatility and correlations among the top few assets in order to create a momentum portfolio with the least expected portfolio volatility, along with a palatable risk profile.

Walk Forward Optimization

January 13, 2014 by Shaun Overton 6 Comments

If you were walking and randomly it started to rain, would you consider carrying an umbrella tomorrow? Of course you would.

The reason I ask a rhetorical question like that is when people observe a behavior, they respond accordingly. If they expect that something might happen again, they change their behavior to accommodate the change in outcomes.

When you think about forex robots, everybody has the dream of developing a strategy that works forever. It requires no changes. The initial settings always work. Turn it on and move to the beach.

Reality, of course, is more complicated than that.

Walk forward optimization continually optimizes throughout time instead of looking for one set of static settings

That leads to expectations of what you need to do when your strategy inevitably goes awry. It’s very possible that you come up with a strategy that works and does amazingly well on the current market. However, a past genius doesn’t mean future genius. There’s always the chance that your strategy will no longer work in the future.

Why is that? It’s the same reason that you might carry an umbrella tomorrow if it rains today. People observe the market performing in a consistent manner. As more and more people make the observation, people start trading on it. The market responds to those changes, and eventually the opportunity completely washes out as too many people have eared about it.

Walk forward testing is the process of determining whether or not your strategy has washed out. By testing on one set of data, and then testing it on a blind set, you can give yourself an indication of whether your strategy is bad or not. The goal of walk forward isn’t to prove that your strategy is good. It’s to prove that your strategy is not known to be bad.

The process of walk forward testing is very simple. You identify a set of information that you want to use for your testing and optimization. Using a real example, right now it’s the beginning of 2014. So maybe you want to look and test data from 2011 through 2012. That would be your in sample data, and then your out of sample data might be all of 2013.

In order to conduct a walk forward test, you would test and analyze your strategy 2011-2012. Then, to determine if it’s “not known to be bad”, you then walk forward to 2103 to see review the performance.

What you’ve done is a blind test. You didn’t know what how the strategy would perform in 2013 when you tested it in 2011-2012. By putting it on a blind sample, you give it the opportunity to fail.

The reason so many traders put their faith in walk forward testing is because it’s the absolute best tool to identify weaknesses in your optimization. When you’re testing a strategy, it is very likely that you’ve overfit to past opportunities.

Self feedback loops in the current market

Let me give you an example. In the current markets, a lot of traders have been banging gold on the market open where every day at market open., they sell as much gold as they possibly can. Sometimes it’s several multiples of the annual production in a span of a few minutes. What you see is an absolute freefall for five or ten minutes. That state persists for days at a time. But that doesn’t last forever. When enough traders start seeing that people bang gold on the open, they start doing the same thing.

Effectively, whoever wants gold to falloff on the market open has taught other traders to do that trade for them. As people expect gold to fall in the first five minutes of the open, they then change their behavior. Some try to jump on banging the open and go short.

Others start modifying their behavior. They notice that gold free falls for five minutes. Then, all of a sudden it stops, and more than like it reverts to the mean. They’ll start changing their tack and buying after so many minutes have elapsed from the open. They expect that the heavy volume that preceded the selling will eventually return to normal. As people change their behavior, other people respond in kind.

If enough people start selling on the open and then buying on the open five minutes later, you can see that a pattern is forming where one person responds to the actions of another. It’s a self feedback loop where the state that was working for the first couple of days no longer works in the future.

If you can identify a strategy that is able to survive those conditions, and is able to survive conditions where you didn’t do any testing and optimization, you give yourself better odds of succeeding in the future. It means that not very many traders have clued into this trading opportunity that you’ve discovered.

The approach to to walk forward testing is the antidote to the problem known as curve fitting. Curve fitting is the ultimate woulda coulda shoulda strategy. It’s akin to opening a chart from yesterday and saying I would’ve bought here and I would’ve sold here, already knowing what transpired.

Of course you’re going to “make money” in that situation. You know with perfect information what the market did. In the future, you don’t know the perfect information. The goal of a strategy is to deal with that ambiguity.

Curve fitting means that you’ve fit everything so perfectly to past market conditions that when new situations inevitably arise, sort of akin to the phrase, “history doesn’t repeat itself, but it rhymes,” your strategy does the same thing.

You want a strategy that does well on past performance, but you’re not coming up with a strategy to make money on historical markets. The purpose of developing a strategy is to make money in future markets. When you’re backtesting, you’re trying to strike the balance between solid historical performance and, most importantly, making sure that that historical knowledge extrapolates to future performance. Your goal is to make money.

Rolling Walk Forward Optimization

Rolling walk forward optimization takes the walk forward idea and continuously improves the strategy by exposing it to new data. So let’s say that you have a twenty four month sample period. One way to go about it would be to optimize your strategy for a period of two months, then to walk it forward to the third month. You observe the behavior and you reoptimize for the second and third month, then walk it forward to the fourth month.

By doing so continuously, you eliminate the decay time of the strategy and give it a chance to adapt to ongoing market conditions. It is sort of the redheaded stepchild to machine learning. Experience and losses give the strategy the opportunity to improve and adjust to the market changes through walk forward optimization.

…you eliminate the decay time of the strategy and give it a chance to adapt to ongoing market conditions

Another important consideration for walk forward analysis is the degrees of freedom within a system. For example, let’s say that you are analyzing a moving averaage cross. You’re using two moving averages and use a fixed stoploss and take profit. That would give you four degrees freedom. The fast moving average is the first degree. The slow moving average is the second degree. The third is the stoploss and the fourth is the take profit.

The more degrees of freedom that you allow in a system vastly increases the chances 0f curve fitting your systems to historical data. The absolute best systems maintain twelve degrees of freedom or less. You want to find trading opportunities that have large numbers of trades and that offer performance that you find satisfactory.

Another element to consider in your optimization is what are you optimizing for. Most people focus on the absolute return. Returns are great, but most traders care much more about how they make their money instead of how much. Let me give you an example. If I had a system that made $25,000 last year, would you want it? Almost everybody says yes.

If I have a system that made $25,000 last year, but you had to lose to $15,000 before you made any money. Most people don’t want that system. What this means is that you care a lot more about the performance on a day-to-day basis rather than end result. The problem with optimization and even walk forward optimization is that you’re not necessarily focused on what you care about in the real world: the way that you’re making your money.

Most charting packages focused on the net outcome and that can cause some weaknesses in your system. If you’re range trading, what you’ve really done is cherry pick the results that are the least affected by substantial news. In effect, you’ve chosen the settings that have not yet been affected by fat tails.

If you’re trend trading, you’ve done the exact opposite. You intentionally pick the settings that maximize the fat tailes that have happened in the past. With trend trading strategies, you probably aren’t going to find consistent performance. Instead, what you’ll find is that the optimization frequently causes long, ongoing droughts of incessant drawdown. Then suddenly, almost out of nowhere, it finds a mega monster winner that returns several multiples of the drawdown that you experienced. This is fine for a hypothetical backtests, but in the real world where you’re suffering losses on a near daily basis, most traders can’t take the pain. The weakness I find with most optimizations is that they don’t look at the consistency of performance. A potential substitute for optimizing a strategy would be looking at the linear regression of the equity curve over time. The best equity curve has the strongest linear regression slope.

Popular charting packages that implement rolling walk forward optimization are Amibroker, TradeStation, Multicharts and NinjaTrader.

Walk forward optimization in NinjaTrader

Open the Strategy Analyzer from the Control Center. Click File / New / Strategy Analyzer.

Open the strategy analyzer in NinjaTrader

Left mouse click on an instrument or instrument list and right mouse click to bring up the right mouse click menu. Select the menu item Walk Forward. You can also click on the “w” icon in the Strategy Analyzer toolbar. If you prefer hot keys, you can also use CTRL + W. Lastly, you can also push the “W” icon at the top left of the Strategy Analyzer.
Select a strategy from the Strategy slide out menu
Set the Walk Forward properties (See the “Understanding Walk Forward properties” section below for property definitions) and press the OK button.

There are many ways to select walk forward optimization in NinjaTrader

The Walk Forward progress will be shown in the Status Bar of the Control Center.

NinjaTrader Screenshot

July 26, 2013 by Shaun Overton Leave a Comment

Taking a screenshot of a chart in NinjaTrader is a 3 step process. You don’t need any outside software or special skills.

Open a chart
Right click. Select “Image” from the menu, then “Save As”
Select the location where you’d like to save the file

Right click on the chart to take a screenshot.

I always recommend saving files to the desktop if you’re not very savvy with computers. You can find them easily.

Comparison: MetaTrader, NinjaTrader, TradeStation

July 19, 2013 by Shaun Overton 15 Comments

Pros and Cons of MetaTrader, NinjaTrader and Trade Station

NinjaTrader, MetaTrader and TradeStation are by far the most popular platforms among the retail trading crowd. Each one tends to focus on a special market niche. MetaTrader primarily caters to forex market. NinjaTrader’s fan base mainly comes from futures. TradeStation earns most of its business from equities traders.

Despite their popularity among certain markets, each platform is fully capable of handling the different types of markets. It all comes down to which platform suits your needs. They all offer automated trading as well as backtesting capabilities. If popularity plays a role in your decision making, then you may want to review Google Trends.

Advantages of MetaTrader		Disadvantages of MetaTrader
Literally hundreds of different brokers offer the platform Large number of commercial products (indicators and Expert Advisors) are available for sale It’s 100% free Everything is setup once you download the platform		No reliable data source for backtesting Allows repainting indicators Slow execution, not suitable for running multiple high frequency strategies MQL restricts you to only trade related programming. Anything else requires DLL programming Can’t keep record of bid and ask in the backtester. No custom time frames on charts MQL itself has a lot of unusual bugs
Advantages of NinjaTrader		Disadvantages of NinjaTrader
Free to use in demo mode, affordable live mode Very scalper friendly with ATM feature Multiple brokerage offerings C# programming offers extreme flexibility Supports multiple charting types Great technical support on their forums Lots of third party add ons are available Ability to securely sell strategies using the native software		The learning curve is fairly steep to get started. The datafeed is not included. Programming usually takes longer than in other platform The number of supported brokerages is far lower when compared to MetaTrader Strategies that are very concerned with position sizing require a lot of extra programming to simulate in the backtester Approval is centralized for all third party products
Advantages of Trade Station		Disadvantages of Trade Station
Great technical support on their forums Simple strategies are very simply to program Lots of third party add ons are available		It’s much more expensive compared with its competitors Can only trade with Trade Station as your broker No demo mode offered Approval is centralized for all third party products

If you would like to develop an indicator or strategy for any of these platforms, we offer conversion services between charting packages.

Things that go wrong programming

April 26, 2013 by Shaun Overton 2 Comments

I started this business to focus on designing better trading systems. Programming obviously plays a large role in the process.

What most people don’t realize is that the programming experience can be quite challenging. When a project takes more time than expected, it tends to take far more time than the original estimate.

Programming is like Air Travel

Many of you travel regularly. Flying is pretty much a given when you travel any significant distance.

Programming is like air travel. Small problems compound into big ones.

How many times have you traveled and the flight arrived 5 hours early? The question is laughable. It doesn’t happen.

20 minutes early to the gates makes most frequent fliers ecstatic. They know that arriving early, even if only by a few minutes, is as good as it gets.

Performance does relate to the airline to some degree. Checking for maintenance problems prevents surprises 20 minutes before takeoff or, heaven forbid, in the air.

The crew arriving on time helps. The last time that I flew from Dulles to Dallas, the replacement crew arrived at the gate an hour late.

The last two times that I flew to Dublin, United Airlines lost my bags… both times. Sometimes, it really is 100% the airlines’ fault.

Force majeure

Those experiences aside, how many times do airlines goof up so badly that travelers arrive days late? Travelers do arrive with severe delays, but those circumstances are usually weather related. It’s outside of the airlines’ control.

I remember the volcano in Iceland that erupted a few years back. People were literally stuck in Europe for a week.

The sequester is a great example. In what’s certainly a willful choice to inflict pain on fliers, the FAA decided to furlough air traffic controllers at major airports.

Those airports are the same ones that I frequent. When I fly to Dublin on Tuesday and I’m potentially 4 hours behind schedule, I’ll be angry. But, I’ll also know to direct that anger at Congress instead of the airline.

Programming and Travel are Fragile Systems

The idea for this article came from Antifagile, where Nassim Taleb discusses how small changes create exponential problems.

Travel is familiar to all of us, so when we think about the delta, which represents the small changes, picture it as the time delay or increase in transit time.

Consider the effects of 3 different deltas

Consider my layover in Newark. How late can I be before I miss the connecting flight. If I miss the connection, how long does it delay me?

20 minutes – The change here is minimal. I will suffer a great deal of (probably unnecessary) stress. My wife and I might jog across the terminal, looking slightly foolish in the process. Nonetheless, the chance of making the connection is near certain.

60 minutes – This is scenario is right on the verge of disaster. My poor wife will listen to me groan and bite my nails as I flip out about missing the connection.

If we do make the flight, it’s only because the airline decided to hold he flight at the gate. Doing so inconveniences hundreds of waiting passengers while a handful of travelers scurry to board the flight.

If they don’t hold the flight, well, then I’m screwed.

The best scenario the can occur after missing the connection is that the airline transfer us to another European destination. The airline then needs to put us on a partner airline to fly us into Dublin, backtracking where we just came from. A one hour delays causes us to

Wait for another European flight
Fly an extra hour to a different destination
Wait for a Dublin connection in a different airport
Fly an hour backtracking

A delay like this could easily result in a an extra 6-8 hours of travel time- all from a 1 hour delay.

3 hour delay – Catching another flight to Europe looks really optimistic. The best case is that the airline put us up in a hotel for the night and sends us on tomorrow’s Dublin flight. A 3 hours delay expands to a 24 hour wait, plus the remaining flight time.

Programming

Ok, Shaun, Ok. What does his have to do with programming?

Just like traveling, a programming project can only go so well. Whenever something unexpected occurs, the problems compound themselves exponentially.

The Evil Delta

Time is the enemy of the traveler. In programming trading robots (or programming anything, really), the delta is the degree of surprise.

Operating system changes: We developed a custom MT4 plugin for a client that likes to trade price ladders. One week after delivering the software, Microsoft released an operating system update. The update broke code in the software that we provided.

Communication: You believe that you asked for one thing, but you get another. Items that seem like minor oversights can blow up into major problems.

Chris worked on a project last month that sought to execute a trading grid at precise intervals. Chris’ original version used market orders. A handful of bugs popped up, but the core of the original version worked well. The client, however, assumed we would use pending orders and requested that it be changed.

The change ruined the original design. More importantly, we discovered that achieving exact execution was fundamentally impossible because we couldn’t precisely control the execution time.

What started off as a 5 hour project blew up to 30 hours of work. The delta from communication surprises is evil.

Basic market mechanics: Sometimes we get asked questions where the trader should know the answer. A common trader-induced question that we get asks why trades suddenly close at market. Traders should have enough knowledge and experience to avoid such basic problems.

The delta on these issues varies, but they’re not as severe as communication issues. They can go anywhere from 20 minutes spent researching the issue to several hours.

Things that can go right in a programming project

Deliver code on time. The time requirements for on time delivery are the easiest to predict. Projects start with a goal. The coder has a good idea for the amount of time required to build a working version.
I view this as analogous to a flight crew arriving on time. The bar is pretty low here.
The code works bug free the first time – no doubt your first response here is, “That’s the way it should be!”. It’s certainly the way that I’d like it to be, but it often doesn’t work out that way.
Most software problems result from communication. When we write a scope of work and program an expert advisor, we believe that we fully understand the requirements.
It often turns out that some of the requirements were not communicated. The product literally follows the order. It’s only when viewing the trades enter the market that the client realizes that they did not ask for something – just like the client that wanted pending orders instead of market orders. They incorrectly assumed that was understood when it was not. The experience of seeing the missing features is the only way the user recognizes the oversight.
Treat people nicely – Programming is a service, but nobody wants to feel like the person on the other end only cares about money. I genuinely care about designing trading systems and helping people. When a customer does business with OneStepRemoved, I want them to trade better and to know that we care about their long term success.
You can always email me personally if you feel you’ve been treated otherwise.

What kind of surprised have you dealt with when programming your trading robot? Share your experiences in the comments section below.

IQFeed Kinetick Trick

March 19, 2013 by Shaun Overton Leave a Comment

NinjaTrader is my favorite strategy backtesting platform. Kinetick works great and is an important part of why I use NinjaTrader – the historical data service makes it so easy to find reliable price history for many instruments.

Collecting good market data poses one of the biggest problems to a systems trader. As I always like to point out, GIGO: garbage in, garbage out. It doesn’t do any good to backtest a strategy on junk price histories.

The same company that owns NinjaTrader also owns Kinetick. It’s no coincidence that the only natively supported data provider is a sister company.

That relationship might lead one to believe that the downloading speed of Kinetick would run fastest inside of NinjaTrader. One would be wrong.

I heard a number of traders complain last week at the Trade Empowered summit about Kinetick’s download speeds. It takes a long time for Kinetick to load historical data, regardless of whether its forex or equities data. Jason, a trader that I befriended, jumped into the conversation with a nice trick that he learned. It’s worthy of sharing with everyone.

Use Kinetick with IQFeed

IQFeed accepts the user name and password for live Kinetick accounts. That in itself isn’t very noteworthy.

What’s important is that IQFeed runs many times faster than Kinetick. If you have a live Kinetick account, download IQFeed onto your computer. Complete the installation process by running the .exe file.

Create a new account connection in NinjaTrader. Instead of selecting Kinetick, select IQFeed. Push next.

Create a new IQFeed connection in NinjaTrader 7

The new screen asks for your IQFeed user details. You don’t have any because you subscribe to Kinetick. That’s ok! Enter in your Kinetick details anyway.

Enter your Kinetick account information into the IQFeed details screen

Enter your Kinetick account information into the IQFeed connection screen.

Name the connection something that you’ll remember. I called mine KinetickIQ.

Now go to the Control Center. Select File \ Connect \ KinetickIQ. When the connection loads, IQFeed will connect using the Kinetick account information.

The download speeds magically run several times faster than connecting directly to Kinetick. I have no idea why this is, but it’s a cool trick if you’re in a hurry to downloads reams of price data.

Did this help you out? Please leave comments in the section below.

Scalper EA

March 5, 2013 by Shaun Overton 88 Comments

It looks like I accidentally stumbled into a scalping strategy. I posted the rules near the top of the page for those that don’t care about how I developed the expert advisor.

Warning: this EA cannot possibly profit on wide spreads. Do not follow this method if your broker’s spread and execution slippage average more than 2 pips.

Scalper EA Trading Rules

Chart: EURUSD 5 minute
SMA Period: 200
Moving Average Envelope: 1.0% of the SMA
Style: Counter-trend scalper

Entry rules

If the price crosses and closes below the lower envelope, then buy at market.

If the price crosses and closes above the upper envelope, then sell short at market.

Exit rules

If the price crosses and closes above the lower envelope, then exit long at market.

If the price crosses and closes below the upper envelope, then exit short at market.

Notice that the scalper strategy uses the same envelope for entry as it does for exit. The distribution of distance around the moving average is sticky when the price extends far away from the SMA.

The screenshot from NinjaTrader shows trades entering and exiting around the lower envelope

Get the code for MetaTrader 4 or NinjaTrader

Why the strategy works

The original intent for this research sought to uncover a range trading strategy based on the price crossing the moving average. Most traders think of distance in terms of pips.

Pips are valid for general context. The problem with modeling a strategy based on pips is that the the meaning of one pip’s movement changes over time.

Taking the idea to extreme lengths, the value of a pip in 1999 when the euro launched around 0.80 hardly compares with today’s price of 1.30. Keeping the discussion to percentages helps to fix the meaning of an idea like 100 pips over long periods of time.

I wanted to visualize how the price generally behaves relative to the moving average. A custom NinjaTrader indicator that my programming team wrote collects and analyzes the data in an Excel spreadsheet. Excel allows me to draw a graph of how frequently the price stretches away from the 200 period simple moving average.

Distribution of prices from SMA 200 on EURUSD M5

The frequency of percentage distances from the SMA 200 on the EURUSD.

The slope of the curve bends as the price extends further from the moving average. As you move left to right along the horizontal axis, the slope is steep until 0.75%. A kink in the curve forms at that point. The slope of the line flattens substantially from that point onward.

A big slope implies that the price will be anywhere but here on the next bar. A flat line means that the price isn’t likely to go anywhere. The distance is sticky at that level.

Scalping in a moving market doesn’t make sense. It’s only when we identify the sticky price condition of 1% away from the moving average that running a scalper EA makes sense.

Scalper EA Backtest results

I developed the strategy in NinjaTrader using data from 2011. My blind period was 2012, which was data that the strategy never saw in development. It was a pure walk forward test.

Results without spread costs

Profit: $5,740 on 108 trades trading 1 standard lot per signal
Profit factor: 2.18
Percent accuracy: 83.33%

NinjaTrader backtest, M5 EURUSD for 2011 without trading costs

Results with 2 pip spread costs

Profit: $1,420 on 108 trades trading 1 standard lot per signal
Profit factor: 1.24
Percent accuracy: 65.74%

A 2 pip spread substantially weighs on the performance

The scalper EA is incredibly sensitive to two assumptions: spread and slippage. I assume that anyone following this methodology trades at a reputable broker with good execution. The backtests assume that the combined cost of both spread and average slippage is 2 pips.

If your broker does not provide execution and spreads within that 2 pip window, then do not trade this strategy. I would not expect for you to walk away a winner.

Walk forward results without spread costs

Profit: $1,960 on 34 trades trading 1 standard lot per signal
Profit factor: 4.38
Percent accuracy: 88.24%

The NinjaTrader backtest shows walk forward results from 2012 on the EURUSD M5.

What is scalping?

Scalping refers to a short term trading style. Profits are very small and occur a large percentage of the time. When losses happen, they tend to be several times larger than a typical winner.

The high number of wins attracts traders of all stripes. The idea of consistently earning profits makes trading more fun and appealing. Traders with experience, which inevitably means traders that have suffered losses, also find the high percentage win rate appealing. It makes the emotional suffering far less difficult.

The emotional component that attracts traders to scalping strategies leads to illogical business decisions. Traders place the need to win frequently above the long term need to expect a profit.

Too many scalping expert advisors tap into the high winning percentage. Most fail to present a clear and obvious reason why it makes sense to scalp.

The EURUSD typically costs $2 to trade a mini lot. Many scalpers set narrow profit targets between 1-5 pips, which are worth $1-5.

The trader spends $2 to make $1-5. If this were a normal business, that would be the end of the game. You win. The game is only limited by the number of trades placed.

Trading, unlike other businesses, frequently results in losses. It’s very possible to build an expert advisor that could win trading for free but loses when costs enter the picture.

The best example is the difference in the scalper EA backtests for 2011. The first test showed a percent accuracy of 83% without including the spread. Adding the spread to the second backtest decreased the accuracy to 65%.

Trading costs make all the difference in scalping. Many scalping strategies live and die based on their trading costs.

The accuracy dropped because all trades had to subtract the spread cost from their simulated winnings. The number of trades that flipped from profit to loss because of a 2 pip spread shows how many trades profited by the narrowest of margins. The narrower the margin of profit, the more sensitive a strategy becomes to spread costs.

Do you think that I should have considered other ideas in the strategy? Suggest some ways to improve in the comments section below.

After-thoughts

This series eventually led to a profitable trading strategy. If you’d like to read through the journey, then I suggest reading the articles sequentially

The initial strategy idea
Selecting an appropriate time frame
A research plan
An annoying surprise in the initial backtests
An attempt at range trading
Range trading results
The moving average envelope scalper

Import a NinjaTrader Strategy

February 15, 2013 by Shaun Overton 17 Comments

It’s easy to import a NinjaTrader strategy or indicator. Popular places for finding a new strategy or indicator include the NinjaTrader forums, Big Mike’s Trading and passing files amongst friends.

The first step is to go to the Control Center, the main screen in NinjaTrader. Click on File, then select Utilities. A new menu will fly out to the right. Select the top option, which is Import NinjaScript.

import ninjatrader strategy or indicator

A screenshot from the NinjaTrader Control Center. Client File, Utilities, Import NinjaScript to bring a new strategy or indicator into NinjaTrader.

Zip Files

Once selected, the program asks you to locate a zip file. You’re probably used to programs using their own funny extensions. MetaTrader, for example, uses .mq4 files for its strategies.

NinjaTrader sticks to using .zip files for both a strategy and indicator. The process to import a NinjaTrader strategy is exactly the same as an indicator.

Select the zip file wherever you saved it. The desktop is always a safe place to download files. If you downloaded an indicator but aren’t sure where you put it, then check the downloads folder. You can find that in C:\Users\YOUR USER NAME\Downloads\. Internet Explorer, Firefox and Chrome all download files to this folder as part of their default settings.

The nice thing about NinjaTrader using zip files is that you don’t see all sorts of funny looking icons. The disadvantage is that it’s impossible to tell whether the file is an NT7 strategy or indicator, or if it’s something entirely different.

If you see a zip file and can’t tell whether or not it belongs to NinjaTrader, the simplest way to find out is to open the file. Double click it.

Exported NinjaTrader zip files contain Info.xml and potentially two folders: Indicator and/or Strategy

When the folder opens, you should see a file called Info.xml. The number of folders that you’ll see depends on the file type. If the zip file only contains an indicator, then only the Indicator folder appears. A zip file containing a strategy will more than likely show both the Indicator and Strategy folders.

Conclusion

Common errors that pop up when importing files are compile errors. Read through that article if you get stuck trying to import a new file.

If you run into any frustrating errors during the process, then please leave a comment on the blog below. I’d be glad to help you out.