Range Trade at High Frequency
Range trading systems make the best candidates for high frequency systems. They are less execution sensitive than trending systems for a simple reason. Range trades “catch the falling knife,” making them suitable for using limit orders.
High frequency prices vary from the normal M30 and H1 charts. The lower the time frame, the better that the chart fits to a normal bell curve. One common theme in systems trading since the 2008 crash has been “tail risk” or “fat tails”, which refer to the edges of a probability distribution like the bell curve. The fatter the tails, the more likely that a range trading system is to crash and burn.
The bell curve shows the tail risk of important events. The tails are colored in red. Fat tails mean that important news happens more frequently
The real world events captured in the tails reflect headline news like Bernanke speaking or Ireland announcing another referendum on all this bailout nonsense. The events only happen once, obviously. If you consider the news events in the context of hourly charts, they happen frequently as a percentage of the overall period. If you look at a one minute chart, that same event is now about 1/60th as important. Dropping down to tick charts nearly makes the events disappear in the statistical profile.
My experience is that the news cycle drives trends on a macro basis. “Macro basis” and high frequency are two topics that don’t belong together. Trending systems should focus on long term trading, while ranging systems are far more suited to high frequency. If your system trend trades, you can throw it in the rubbish bin for high frequency trading ideas.
High frequency considerations
Keep in mind that there are effectively two ways to participate in the forex market: you can either act as a price taker or as a price marker. Price takers range across all market participants. A hedge fund or university endowment is just as likely to take a price as they are to make one. CTAs and retail forex traders are much more likely to make their decisions based on the expected market direction. Timing is critical for them, so they don’t want to leave it to chance whether or not they’ll get to enter a trade.
The trader gets filled right away. That’s the major advantage. The main disadvantage to acting as a price taker is that you pay the spread every single time that you want to enter a position.
I sat with AvaFX in Dublin on my last trip. They charge a 3 pip fixed cost spread. I mentioned my concern about how that spread affects my client’s EA performance. His MetaTrader expert advisor trades 4 times per day on 2 currency pairs. If you do the math on a 3 pip spread, it works out to 8 * 260 = 2,080 trades per year. If you’re paying 3 pips and trading a $10,00 account, you would have to earn $6,240 per year – a 62.4% return, just to cover trading costs. I don’t care how good a system is – it will never cover those kinds of costs. Trading on margin will not do anything to resolve the issue. Spread costs are directly proportional to the amount traded, which impacts the profit. There is no way to trade and make money if the transaction costs are too high.
Designing an expert advisor is difficult enough, but it’s even harder when you factor in the trading costs. Say, for example, that I develop a EA that wins 75% of the time with a payout of 0.5:1 before trading costs. When the EA wins, it earns $0.5. It loses $1 whenever a loss occurs. The profit is 75 wins * $0.5 = $37.5. The loss is 25 * $1 = $25. The expert advisor’s profit factor is 37.5/25 = 1.5.
That should sound great. The problem occurs when the total commission outweighs the total expected profit. This example required 100 trades. Let’s say that we were trading mini lots with an average win of 5 pips and the average loss of 10 pips. That puts the gross profit at $375 and the gross loss at $250. The return is $125 for the 100 trades, excpet that we must now subtract the $100 for trading costs. The total profit plummets to a measly $25.
If the expert advisor’s expectations held true for something like a 10 pip take profit and 20 pip stop loss, the trader might be better off to change the exit points. The reason is that the profitability may actually improve. The goal would be to reduce the number of trading opportunities with an eye towards making them more profitable relative to the costs.
A better approach, in my opinion, would be to switch over to market making. Although you usually still pay to trade, the advantage to market making is that you earn the spread rather than paying it. The spread is overwhelmingly most traders biggest cost. Not paying it opens the possibility of applying the strategy where one normally could not afford it.
Market making only works if your forex broker allows you to post best bid/best offer and have the price reflected on the screen. Most brokers claim that they are ECNs. A real forex ECN allows you to post limit orders. Whenever that order represents the best bid or offer, the price and size of your order shows up on the screen. The only retail trader friendly brokers that I know of are Interactive Brokers and MB Trading.
I ran my NinjaTrader license at MB Trading last week to test the execution and order flow. The test only use traded a microlot (0.01) and posted best bid or best offer on the EURUSD. The orders remained valid for anywhere from 1-10 minutes. Despite the small trade size and lengthy time period as best bid/offer, the orders only filled 75% of the time. That meant that I caught 100% of the losers but only 56% of the potential winners. Not good, in spite of getting paid for the limit orders.
Interactive Brokers is the next test candidate. They have been around much longer and should have far more order flow. I’m hoping that the low fill rate that I experienced making a market at MB Trading will improve substantially when I shift the same strategy to Interactive Brokers.
I expect to find a few other changes as well. The spread that I earn should fall from around 0.9 pips on EURUSD to 0.5 pips, which is indicative of Interactive Brokers’ improved pricing. I also will have to pay a 0.2 pip commission, which reduces the net credit from 1.0 pips at MB Trading (0.9 spread + 0.1 commission) to 0.3. Nonetheless, I expect the improved fill rate on winning trades to work more in my favor.
The thing that most people will hate is that you can only test a market making approach with live money. It’s sufficient to backtest a strategy using market orders with a 0 spread assumption. The goal is to weed out the junk from diamonds in the rough. No method exists, however, to accurately determine whether or not a trade would have gotten filled with a limit order. The only way to find out is to test an idea with live money, then to compare the results to a backtest over the same period. If the live, high frequency performance is similar to a backtest, then you probably have a winning approach.
The real motivation here is to get as many opportunities as possible. Just like the casino does everything to help you pull the slot machine faster, the trader should look for as many favorable setups as possible. High frequency stands out in this area. The inherent advantages of a system are more likely to manifest more quickly. Assuming that you get a handle on the trading cost problem, the profit is often limited only by the number of trades that can be squeezed into a day.
Programming options at high frequency
MetaTrader 4 is not a good candidate unless you expect to post orders once per minute or slower. MetaTrader suffers from the Trade Context is Busy error. Running an expert advisor on more than a single instrument could cause orders to enter too slowly or not at all. MetaTrader is only an option with MB Trading. Interactive Brokers does not support MetaTrader.
NinjaTrader works great and offers a lot of the broker portability that comes with programming in MQL. Programming a high frequency strategy in NinjaTrader works at most human speeds (5 seconds or more). For the brokerages where NinjaTrader submits orders using the broker’s API, I find a speed bump affect at work. NinjaTrader processes the orders lightning fast, but the broker API cannot handle the speed and starts to choke. If you want to test any frequency that’s not ultra high frequency, I recommend programming in NinjaTrader.
The FIX Protocol is the best option for the institutional trader that cares about maximal performance and does not suffer from the usual budget constraints. FIX is a fancy way of controlling communications between a custom platform and the broker. It does not involve software, only rules. The FIX protocol allows the trader to write software 100% from scratch. The trades and orders can go out the door literally as fast the machine can process them. It’s the advantage that comes with building everything from scratch.
Optimize an Expert Advisor
One of the lesser known features of the MetaTrader backtester is the optimization feature. It’s so small that you could be forgiven for overlooking it.
Optimization is the process to maximize a certain outcome. In this case, it’s profit. Any EA developer wants to maximize the amount of profit made over a given period of time. The MetaTrader optimizer allows the trader to search for the combination of inputs that yielded the maximum profit over a given period of time.
The process is identical to running a backtest, except that MT4 runs multiple backtests at the same time. It then organizes the results and offers up the best combination.
Telling the backtester to run in optimization mode is easy. Simply put a check next to the word Optimization. MetaTrader will then sort through the combinations that you tell it to consider.
Place a check in the box next to Optimization in the MT4 backtester
The next step is to click on the Expert properties button to the right. A new window appears that contains three tabs: Testing, Inputs and Optimization. These screens allow the trader to inform MetaTrader which variables to consider for testing and how to weight the results.
Testing
The top of the testing section applies to every type of backtest. Here you can select the starting balance. MetaTrader defaults the option to $10,000, although you can make this any amount of your choosing.
The second default option allows the trader to restrict the direction of trades. It’s a frequent expert advisor programming request. It’s also one that is unnecessary. Both the backtester and expert advisor options screen allow the trader the option of restricting trades to long only or short only without additional programming. If the EA is not well programmed, this setting may cause errors 4110 or 4100 to appear all over the trading journal. It’s harmless. The only effect should be that the backtester slows down. It’s the result of writing to the journal hundreds of times or more.
The testing tab of the MetaTrader backtester
A groupbox appears underneath these options that inexplicably relates to the optimization process. You’d think it would make more sense to place it in its namesake tab. That’s typical MetaQuotes logic at work.
The first line contains numerous parameters for choosing the best option. User overwhelmingly select for the largest account balance, but other options include the profit factor, expected payoff, maximum drawdown and drawdown percent.
The last line automatically uses a genetic algorithm. Optimization processes use either brute force methods or genetic algorithms. Brute force strikes most people as intuitive although obviously exhausting. The software tests every combination possible. Genetic algorithm’s attempt to make the process more intelligent. When the software sees that certain parameters almost inevitably lead to a losing performance, the algorithm skips similar tests where it expects to lose.
This is a great idea if you have a quality genetic algorithm. My opinion of the MetaTrader backtester is less than stellar. I don’t feel very confident about the algorithm at all. If you don’t mind spending extra time waiting for test results then I suggest unchecking this option. You don’t want to miss a potentially important combination.
Inputs
Most people find this screen confusing. The first column, called value, strictly controls inputs for simple backtests. The Value column is totally ignored during an optimization run.
The inputs tab of the MT4 backtester expert settings
The important columns for this task are Start, Step and Stop. Start is the lowest number that the backtester will consider. Step refers to the interval between the lowest value and the highest value. Tightly controlling this setting allows the user to gain quick insights into how changing the variable values affects performance without dragging the tests out for a full week. Stop is the highest number that the expert advisor will use.
The most obvious candidate for testing in this example is the Take Profit value. The default setting is listed at 50. If you trade the majors, you might want to consider settings ranging between 10 pips and 200 pips. That means that you set Take Profit row to 10 for the Start column and 200 for the Stop column. The real trick here is selecting the Step. If you choose Step = 1, then MetaTrader will run a separate test for every value between 10 and 200. That’s 190 tests, which is overkill. A step of 10 cuts the total number of tests down to 19.
Optimization
This section is the nit-picky part. If a trader feels it’s unacceptable to have 10 consecutive losses in a row, he can place a check next the the Consecutive wins box. MT4 automatically discards any tests which yield a result that contains anything checked off.
The optimization tab in the MT4 backtester allows users to discard tests with undesirable traits.
When you finish going through each of the tabs, push OK in the bottom right corner. It’s time to launch the tests.
Curve fitting in the MT4 Optimizer
A word of warning: my personal opinion is that optimizing an expert advisor is usually a very bad idea. The unique settings that yield the most profit in 2012 are unlikely to yield the most profit in 2013. If you don’t control for random chance, there’s a good probability that the 2012 best combination may result in catastrophic losses in 2013.
I recommend that traders pursue any strategy development work in NinjaTrader. I don’t like the idea of optimizing at all. Instead, I always focus on testing strategies for entry and exit efficiency. I know from years of experience that these values never fundamentally change on instruments of the charts traded. Entry and exit efficiencies make wonderful metrics for automated trading because they are so stable.
Volatility & Divergence Commentary
This week has been an ideas week. An unusual number of clients are asking for my opinion on the ideas that they want to program into an expert advisor. Divergence and volatility keep popping up as themes for the week.
Simple Volatility Filter
Volatility is one of those factors that you cannot ignore in trading. It highlights the overall risk context of the market and says something about the likelihood for a trade to get some wheels.
The number of tools that we have to study volatility is unfortunately very limited. Almost everyone uses ATR, which is the average true range. The calculation for it is very basic. The true range is simply the high minus the low. The ATR is simply the average of all the true ranges over a certain period. Most traders use a 14 period ATR by convention.
I sent the chart below to a client in Australia yesterday who asked if I had any ideas for a volatility filter. It compares a fast and slow volatility window using ATR. The red line represents the 14 period ATR, which I call the fast line. The blue line represents the 300 period ATR, which I call the slow line. I suggested that period he could use the fast line appearing above the slow line as an indicator of high volatility. The opposite indication would indicate low volatility.
Two ATR lines may signal a trend, although they would not indicate the direction.
I created the above chart by dragging and dropping the ATR custom indicator onto a chart. I then dragged and second ATR indicator onto the first ATR indicator. Doing that way overlaps the lines. 0therwise, you would see two lines in separate windows.
When I opened MetaTrader again this morning, the same chart was left open. I immediately noticed that the line crossings appeared to match up with some of the longer term trends. Although it would not indicate the direction of a trend, the ATR crossings might prove useful as a trend detection indicator. If you decide to research this idea, please leave your comments and observation on the blog page below. I enjoy hearing from my readers.
Divergence
I buy into the idea that the market contains price points that are more relevant than others. A lot of the math that I work with involves autocorrelation, which many refer to as the long term memory function. It’s a mathematical tool that allows nerds like myself to find hidden statistical patterns among a bunch of noise in a signal.
Divergence takes a similar idea and applies it to indicators, the most common of which are the MACD, RSI and stochastics. When the price rises above a previous critical point and the indicator does not exceed its previous critical point, then divergence exists. Most traders claim that divergence signals the potential end of a trend.
My biggest gripe with divergence is that the length of trends exhibit random periods. I’ve done plenty of independent research on this topic. Regardless of the method that you use to pick market tops and bottoms or how you define a trend, the time period of the measured trend is always random. It has a probability density, but it definitely does not have a set number.
Divergence completely fails to address this concern. There’s no reason why you can’t have 2 divergences or even 5 divergences in a trend. Divergence does not help the trader distinguish between the end of a trend or a continuing trend. You could use divergence as a trend detection tool, but by that point some traders are already calling for it to end. My personal opinion is that it’s not very useful.
My other complaint with divergence is that the method for picking critical points is totally arbitrary. If you put 10 traders in a room and ask them to draw a trend line, you will get 10 different answers. The absence of consensus on such a basic concept ought to say a great deal about the value of subjective interpretation.
Traders also attempt to draw the points between swing highs and lows. That task should be obvious, but it’s not. I always recommend using the zig zag indicator when customers want to go down the swing trading route. They quickly discover the same problem – how sensitive should the settings be. Again, we circle back to the issue of period length. The swing high that Bob’s Zig Zag settings draw looks like market noise to the swing highs that Alex draws.
My opinion is to stay away from divergence and look for other techniques.
Market Depth
Moving from small time retail forex accounts to a serious account size comes with some bumps in the road. Most traders see the prices of forex pairs on the screen and assume that they can buy and sell unlimited quantities at any time. Although the forex market is the largest market of any in the world, making it the most liquid, there is still a limited size to what you can trade at any moment.
Reading Forex Market Depth
NinjaTrader and MB Trading both make market depth information available in their trading screens. It shows where all of the nearby liquidity lies. Say, for example, that you wanted to place a big order for EUR/USD. You can see in the screenshot from MB Trading’s platform that the liquidity gets bigger as you get further from the price. They call these “Level 2″ quotes, which is jargon taken from equities traders.
MB Trading's Navigator shows the market depth of the EUR/USD. Light colors show the best prices, darker prices indicate distance from the best price.
I took this screenshot in the late afternoon when liquidity is at its worst. The best bid shows a depth of 200, which is measured in mini lots of 10,000. You could sell up to 2 million EUR/USD and get filled at that price. Notice, however, that most of the liquidity is further away. 3 million sits at the next best price and another 3 million even further from that. The net available liquidity is 8 million on the short side and 6 million on the long side for a total of 14 million.
The FX Pro screen in NinjaTrader makes it even clearer. I took this screenshot several minutes later, which is why liquidity numbers are different. One thing I really like about this screen is that NinjaTrader converts the liquidity depth into more tangible numbers. The formatting makes more sense to me. Plus, it’s much easier to keep track of the varying spread.
The FX Pro screen in NinjaTrader
What you learned in economics doesn’t apply to trading. Dealing in bulk actually leads to worse pricing instead of improved pricing. The reason is that forex instruments are so standardized that there’s effectively never a lack of customers. It’s really an issue of getting how much much you want at a certain speed.
Only a fool would hit the market with 20M EURUSD instantaneously unless you’re desperate to exit a position. If you push a market order judging solely from the quote on the screen, you may get 2-8 million filled at the displayed price. But, the rest of the order will get filled at progressively worse prices. The traders making a market want their pound of flesh for letting someone into the market so quickly.
Traders cannot see the liquidity depth of most brokers because they elect not to show it. Their platforms encourage the buy this, buy that psychology. The more information that they broadcast, the more bandwidth that’s required, which means that better servers are required.
The EURUSD is the most liquid forex pair in the world. This varies largely by broker, but at any given time you should be able to trade 20-30 million EUR/USD. That doesn’t mean that you’ll get filled at the price on the screen. What that means is that that is the sum quantity available at any given moment.
Some brokers hide their quantity. It’s not like the stock market where if there are 15,000 shares of Microsoft ready to trade at any given moment, you can see 100% of the liquidity available. In forex, as an OTC market, the broker may wish to restrict the viewable book for a few reasons.
If you offer markets from say 5 banks, it’s very rare for the broker to feed the best competitive price and to let the banks fight it out? Why? Because the broker also needs the banks to stick around when nobody wants to trade.
It’s an informal agreement that if I’m retail broker A and UBS is my main liquidity feed, I go out of my way to give UBS the good flow. My customers expect to trade during NFP and other volatile markets (although they really shouldn’t!). If the brokerage simply lets the banks fight it out, then the banks have every reason to let the brokerage rot on the vine when their customers want to trade but it’s bad for the banks. The banks certainly don’t want to take positions in volatile markets. Their only incentive for doing so is if the “good flow” that the forex broker sends during normal markets incentivizes them to accept the risk of losing more than they care to during NFP.
News traders are the most likely to try trading during a thin market. They are also the most likely to complain about not being able to trade. These are the arguments to which I’m least sympathetic. If you’re trading the news, you are overwhelmingly likely to have less than one year of trading experience. The decision has nothing to do with researching systems or evaluating whether or not it’s a good idea. It has everything to do with gambling.
Retail traders are the most likely to trade during volatile events, not just news but really any type of momentum. Almost everyone follows a breakout or momentum strategy. It has everything to do with what traders perceive as the most likely outcome. When the market explodes in one direction, it takes nerves of steel to stand in front of the freight train. Therefore, it’s probably a good idea because it’s counter-intuitive.
Trends happen so slowly that they don’t excite the gambling buzz that most retail forex traders are after. My friend Afshin in Dublin fell victim to this last week. He saw the EUR/USD rising day after day after day. He felt like it was simply overdue for a correction. The urge to participate, rather than coming from a desire for a quick hit, instead came from a desire to be right before there was any clear indication of the opportunity to be right. The point is that what feels natural to do is often precisely the wrong thing to do. It feels natural to every other trader, too.
Market Depth and Direction
One research project that I’d eventually love to do is to study how market depth on any given side of a market affects direction. Some traders run simple liquidity businesses where they receive trading rebates in exchange for accepting the risk of holding a position over the short run. These entities are less likely to concern themselves with picking the direction of the market.
Trading desks that make markets, however, often want the flow so that they can establish a position and earn the spread while doing so. These entities are picking direction – and they are backed by very intelligent math geeks with PhDs and a lot of time on their hands. If those desks make a visibly deep market and it’s sufficiently one sided, then it’s probably safe to assume that they expect to the market to move in the opposite direction.
When you’re buying a forex pair, the bank is selling it to you. So if everyone stacks the liquidity so that you can buy but the liquidity is thin on the short side, it should be telling you that the smart money wants to go short right now.
High Frequency Forex Seminar
One exciting opportunity popped up while I’m in Dublin next week. Best of all, it’s free and open to the public. If you’re in the neighborhood and would like to discuss trading in person, I’d love to meet you.
Trinity College Dublin invited me to present a graduate level seminar to MSc students in Finance and Alternative Investments on Wednesday, February 8, at 6 pm. The seminar will be hosted in the MBA room, which is on the second floor of the business school. The topic will be high frequency market making in forex.
Topics for the high frequency forex trading seminar (about 10 minutes per subject):
- Market making versus price taking
- Comparing frequency to expectation. The more you trade, the more you make
- Liquidity risks and self-feedback loops
- Technical approaches and limitations
Trading Time in Programming
The major automated trading platforms such as MetaTrader 4, NinjaTrader and TradeStation all count time in the same way. This makes it quite convenient for ordering trading strategies and expert advisors; you don’t have to do any mental gymnastics to describe the strategy in different platforms. The consistent arrangement of time makes it easy for us to translate trading strategies across multiple platforms.
We tend to think of time as moving in the same direction as when we read. English speakers, who read from left to right, think of time as moving the same direction. If you speak a language like Arabic that reads right to left, you tend to think of time as marching to the left.
All of these charting platforms are written by speakers of left to right languages. The past is anything that’s not on the far right side. The present is the square on the far right. Each square represents an equal time interval. Traders know these as bars.
A visual display of time and how it's segmented
What tends to confuse everyone ordering expert advisors is that even though time marches to the right, trading programmers count the bars to the left. What makes things more confusing is that programmers always start counting from 0 instead of 1.
Even though time moves to the right, we count it from the right and move back left
If you want to trade a moving average cross strategy that waits for the bars to close, what you’re looking at is “bar 2″ and “bar 1″. The way I describe this in the scope of work is the value at two closed bars ago and the value at the last closed bar, respectively.
An expert advisor that uses closed bars ignores bar 0 because it is still open, which causes the moving average values to fluctuate. The only way to know for certain a moving average value at a particular bar is to wait for the bar to close, which means it is no longer bar 0. When a client requests an expert advisor that trades intrabar, they intend to compare the moving averages at “bar 1″ with “bar 0″. In plain language, that means to compare the value at the last closed bar with the currently open bar.
Hopefully, this description makes sense when you open a chart and see the bars already loaded. The final confusing element is when a new bar pops onto the screen. The previous examples showed 5 bars on the chart (bars 0-4). When a sixth bar pops up, the count is reset to the new time period on every update.
Say that we’re looking at an H1 chart in MetaTrader and that the current time is 06:00. When the new hour strikes, the chart loads a new candle to represent 07:00. It’s at this time that the count resets.
When a new bar appears, your charting platform resets the count based on the newest bar.