Algorithmic and Mechanical Forex Strategies | OneStepRemoved

Resetting my demo testing

by 6 Comments

I mentioned in my new strategy post that live demo testing would last for two weeks unless bugs popped up. Bugs did pop up… but only in my rush to get ready for Thanksgiving!

I sat in my office frantically trying to wrap up code before business hours ran out on the day before Thanksgiving. The plan was to unplug for the long weekend. And, I did. Emails went on autoresponder and, aside from one night binging on Netflix, I didn’t see a screen for 4 days.

Rushing while problem solving, and especially programming, almost never ends well. I would never push untested code changes onto on a live account. But since I figured the change was simple and it’s only a demo account, it’s not a big deal if I mess it up. There went 4 days of live market testing down the drain.

The bug

I’m testing my live demo on FXCM US. As most US traders are aware, the FIFO rules imposed on FX brokers can get pretty annoying. I noticed that a small handful of my orders were being rejected in a special circumstance.

Say that I’m in a long trade and place my take profit at 1.0550. Now say that the signal is so strong that not only should I exit long, but I should also go short at 1.0550.

Anyone in a normal market would place a limit order to sell double the open position at 1.0550. That would cause the long trade to exit and a new trade to open in the opposite direction.

FIFO rules prohibit this because I’m already long. My intended bug fix was to place a take profit order to exit. The order to go short would then remain hidden in the code until the long trade exits. Doing the order logic this way follows my original intent, but also complies with FIFO.

The buggy bug fix

Using my long and then going short example, my long sets a take profit and then the code was intended to send 1 limit order to go short. I forgot to tell the code that it had already sent that one order. Instead, the code sent duplicate orders on every single tick.

Position sizes of 1 microlot mushroomed into several standard lots. Needless to say, the profit and loss began to swing wildly. The massive jump in size also swamped the statistics that I was hoping to measure with the data gathered so far.

It’s easier to start a new demo account, which is what I’ve done as of 5 minutes ago. One upside to using a new demo account is that I can run the code on an account size that matches my intended live testing balance. I plan to test with $2,000 instead of the $50,000 in the first demo.

The first few days of trading

equity curve

This was the first 3 days of trading

The first few days continued much in the same vein as in my original post. I’m looking forward to collecting data from the new demo account. You can expect to stay up to date on my progress if you’re a subscriber to my free newsletter.

How I picked my brokers

Using limit orders is extremely price sensitive. If you place a limit order to buy EURUSD at 1.0550, the price must exactly hit 1.0550 in order for that trade to execute.

One danger of trading on marked up spreads is that you don’t know whether the broker is widening the bid or the ask. If the interbank market quotes a 0.3 pip spread on EURUSD at 1.05480 x 1.05483 and the broker usually charges a 1.3 pip spread, then the broker can mark up the spread in several different ways.

  • Add the 1 pip markup entirely to the ask. The price on your screen appears as 1.05480 x 1.05493
  • Add the 1 pip markup entirely to the bid. The price on your screen appears as 1.05470 x 1.05483
  • Split the 1 pip markup across the bid and the ask. The price on your screen might be as 1.05475 x 1.05488

I of course have no idea which way the broker marks up the spread. If they’re smart, they will price shade in order to earn extra income from the easy order flow. There’s always the risk that the markup could cause one of my orders to not get filled, even though the real interbank market actually touched that price.

I’m doing my demo test at FXCM for three reasons.

  1. Commission only trading. It’s a pure interbank feed, so I don’t have to worry about how spread markups affect my execution.
  2. The commissions are very fair for a retail trader. It works out to $60 per side per million, which is only double the commission that a small institutional trader would pay. Retail rading costs have fallen dramatically in the past few years.
  3. Seer is already plugged into FXCM

When it comes to trade live, I’m planning to trade a personal account at FXCM US and a second account denominated in euros for my Irish company, Dominari (the same one that owns QB Pro). Dominari’s trading will go through Pepperstone, which offers the same commission only setup but 1) charges even lower commissions in euros and 2) offers much higher leverage.

Order book Slope

by Leave a Comment

I used to make levered bets in the option market during earnings season. Pick a direction, choose an option near the money and close to expiration, then hope. It seemed just as stupid as buying a plane ticket to Las Vegas and betting a black chip on red. By the time all expenses were weighed, I was always charged a premium for entertainment.

I have since learned a new orderbook measure that could provide an edge. It’s an idea that has been around for some time, but has plenty of room for optimization. The math is somewhat cumbersome, but I though I would take a few posts and explain why I might cancel my future trips to Vegas.

Limit Order book Information

When collecting data for an algorithmic trading system, the two most common pieces of information are quoted price, and volume. That said, it would be silly to think that information contained in the limit order book contains no excess details toward the price formation process. Not only do you know the current price, but you also know the current price that others are demanding! Creating a solid model of the order book is the key to understanding aggregate liquidity and trading interests of market participants.

Research in order books has found a significant relationship between volume, volatility, and a value known as order book slope. The ideas are somewhat nontrivial on the surface, so I want to summarize the key points and mathematics for easy implementation.

Order book Slope

If you Google around, the correlations between volume and volatility (typically denoted as the “Volume-Volatility Relation”) are used for several liquidity measures in the market. The Volume-Volatility Relation has empirical findings in stocks, bonds, currencies, and futures, though I will stick to equities in this post for demonstration.

To apply Volume-Volatility to order books requires a notion of slope in a limit order book. Consider the following example.

Orderbook Slope

Two different types of stocks show very different slopes in their order books. The idea of a line connecting the dots intuitively explains the idea of order book slope.

In a market frozen in time, each circle in this example represents a value in the limit order book waiting to be consumed by a market order. In a simple equity market, each tick on the x-axis represents one penny movement in the stock price, while the y-axis represents the total aggregate volume on each side of the market.

The main observation you should be making is the shape of the volume in the market for these two hypothetical securities. The new IT firm seems to be more linear in creating a v-shape, while the blue chip firm has more curvature as it flattens out at the top.

The intuition is there too. Really, take a hard look at Figure 1 and think about it. Blue chip companies like Apple and Google have great volume and perhaps a tighter consensus of price during a typical day of trading. Therefore, it is reasonable to think that just about 100% of the limit order book volume occurs within 5 ticks of the current market price.

On the other hand, think about those penny stock start up companies we have all been enticed to buy at some point of our trading career. Companies with unknown valuation and future potential will have limit order book volume spread ten or more ticks from the current market price. Some people think these companies are doomed to tank as a product of price inflation, while others may own the product produced and realize the potential as early investors. The two diverse opinions will have valuations that come out drastically different and causing the plots above.

Aggregate Orderbook Slope

Now consider what happens over time. If you collect data over time and take an average, the result will appear as in Figure 2.

Aggregate Limit Orderbook Volume

Order book slopes published by Naes and Skjeltorp

Here, about 50% of volume in the blue chip firm has limit prices within 5 ticks (left) while only 10% of volume is within 5 ticks for the young IT firm on the right.

The published research finds a negative relationship between (average) order book slope and the variation of analysts’ earnings forecasts. This means that as the slope becomes more linear (right plot of figure 2) investors disagree more about the value of the firm and thus higher volatility will likely result.

Creating a solid model of the order book is the key to understanding information about aggregate liquidity and trading interests of market participants.

In fact, the order book slope has been shown to have relation to volatility, volume, and the correlations between volume and volatility– all three of which could be profited on.

Surprisingly, in the mid 1990’s, people were studying the shapes of these curves and relating them to the different liquidity providers present in the market. The shapes were shown to find the probability of informed traders completing market orders at any given moment! See the Glosten model for more info.

Making a precise definition of order book slope requires a blog post of explanation of itself. The math looks daunting at first glance, but as I will explain later, has great intuition, elegance and simplicity. Along with that, the formulas are tangible and should leave you with a relatively simple programming problem (assuming you have the data)

Support and Resistance Metric

by 2 Comments

Converting back of the envelope ideas to trading algorithms is extremely challenging, but requires significant attention. This could be the single item that makes your system great. In this post, I suggest a way to find more defined support and resistance metric using limit order books.

Creating a Order Book Metric

To start, I want to create a frozen moment in time where two order books appear as in the following picture. For the sake of this example, I assume other signals have detected a $20.00 resistance level.

Support Resistance Metric Limit Order Book Comparison

Figure 1: Comparison of two limit order books during a suspected $20.00 resistance level.

Blue represents the limit bid and orange the limit ask. The order book on the left suggests strong resistance at 20, while the order book on the right suggests weakness . If you’re not convinced, see my post on the support and resistance in the limit order book.

The idea above is great in theory. How can you quantify the above two situations so that it’s actually useful? Otherwise, it’s just another fluff technical analysis piece with no real substance.

A successful method applied to Figure 1 should determined if a short position should be entered at $20.00 (left) or not (right).

The first of two decisions that you must make is how far to look into the order book — a value I will call N. The above two order books have N = 4 because each side of the book contains orders at 4 different price levels.

Of course this will depend on your access to data. However, you might also find through back testing that increasing N provides no useful information.

The next decision you have to make is how to weigh the volume in the order book.

You might be foolish to think that each level of the order book is created equally.

For simplicity, I am going to describe a linear weighting system where ticks closer to the price midpoint are given more weight. Referring to the first order book example, N=4 implies we need four weights on the ask side. My system has weights that satisfy the following conditions:

  • w1 – corresponds to $19.99
  • w2 – corresponds to $20.00
  • w3 – corresponds to $20.01
  • w4 – corresponds to $20.02
  • w1+w2+w3+w4 = 1
  • w1 > w2 > w3 > w4

These weights essentially alter the volume in the order book so that overcoming the level of resistance is a more pronounced event. Here’s how you can choose them for a linear case.

  • w1 = N = 4
  • w2 = (N-1) = 3
  • w3 = (N-2) = 2
  • w4 = (N-3) = 1
  • total = w1+w2 + w3 + w4 = 10

Through a process of normalization, the final weights are found as follows

  • w1 = N/total = 4/10 = .4 = 40%
  • w2 = (N-1)/total = 3/10 = .3 = 30%
  • w3 = (N-2)/total = 2/10 = .2 = 20%
  • w4 = (N-3)/total = 1/10 = .1 = 10%

Notice that the whole system comes by simply defining N, thus is easy to generalize and back test in your trading system.

The next step is to compute the average weighted price that would have to be paid if a market buyers chooses to pass the level of resistance. Again, I will refer to figure 1 on the ask side of the market. This will show the difference between a strong level of $20.00 resistance (left ask book) and a weak level of resistance (right ask book)

Support Resistance Metric Limit Order Book Comparison

Figure 1 repeated

Average price (left ask book) : $19.99*2+$20.00*8+$20.01*4+$20.02*5/19 = 20.0063

Average weighted price (left ask book) : $19.99*2*w1+$20.00*8*w2+$20.01*4*w3+$20.02*5*w4/(19) = 20.0022

Average price (right ask book) : $19.99*2+$20.00*8+$20.01*1+$20.02*1/12 = 20.0008

Average weighted price (right ask book) : $19.99*2*w1+$20.00*8*w2+$20.01*1*w3+$20.02*1*w4/(12) = 19.9989

From the weighting system, you can see that a assumed resistance of $20.00 is passed by only paying an average (weighted) price per share of $19.998 during a time of weak resistance.

The weighted average price shows more defined support and resistance levels

Generating Signals

Finally, to generate a solid metric, I considered the percent movement a stock would have to take to move past a weighted average price. For example:

Left ask book (strong resistance) : 100*(current price – average weighted price)/current price = 0.11%

Right ask book (weak resistance) : 100*(current price – average weighted price)/current price = 0.094%

My rule is set so that if the current price must move over 0.10% weighted average order book price (for a given value of N) then the resistance level is strong. Less that 0.10% makes me think the resistance is weak, and would signal for me to not initiate a short position.

Support and Resistance

by Leave a Comment

The Limit Order Book is lurking behind every price tick in every market you can imagine. From the ill-liquid real estate market, all the way to high frequency bond trading, the limit order book determines all price movements.

A simple example in my last post of supply and demand demonstrated how price changes in an equity market. I made several arguments justifying the existence of a bid ask spread, and showed how this leads to price formation. My goal in this post is to find clarity in the foggy world of support and resistance using limit order books. Support and resistance information can be used to build confidence when entering or exiting a trade.

Imaginary Price Levels

If you try searching for support and resistance, a wealth of information can be found usually in the form of some article accompanied by several charts with lines claiming to have found the magical levels.

Every such chart I have found, however, has one single thing in common. The stock price always, at some point, just slightly crosses these horizontal lines. To me, it feels like a slap in the face… the ultimate I told you so from market experts that can apparently see into the future. Here’s a great example I picked up on Google.

Support and Resistance chart

Figure 1: Support and Resistance levels breached leaving question of imaginary lines.

I realize that mathematical definitions of these levels exist, and I realize that human psychological traits are often correctly considered. Nevertheless, they still lack precision! The arguments for true black and white support and resistance levels always must always have a fair amount of uncertainty. Sadly, this is just another part of working in this business.

If I see a price move past a resistance level in real time (i.e., not being able to see the entire future nicely displayed in front of me like Figure 1), I often question if the level has been breached. This could perhaps leave you in the worst possible position as the market rockets the wrong way. The limit order book can reduce this uncertainty by displaying real information.

If you try searching for support and resistance, a wealth of information can be found usually in the form of some article accompanied by several charts with lines claiming to have found the magical levels.

Suppose a stock is testing a human psychological resistance level of $20.00 and your algorithm has signaled that you initiate a short position. You wish to enter the market at the highest possible price without missing the peak. More importantly though, you wish to confirm a resistance level still exists . If the current order book is displayed as below left, you would have confidence that enough sell pressure is present to hold the resistance.

Support and Resistance limit order book

Figure 2: The left side of the image shows more market depth on the offer (orange), which is resistance. The right image shows light depth, which is the absence of resistance.

On the other hand, if the order book is displayed as above right, it it would take only a moderate collection of market order buyers to break the $20.00 level– and break it fast in this example. Short sellers would run to cover, and the market could swiftly move against you.

Measure Support and Resistance

I found some research out of Wharton suggesting an order book metric (cumulative depth), and have heard more advanced ideas shared in my personal research symposiums. That said, I think this situation is being made too complex.

Translating the above example into math should be straight forward, and customizable to the strength of signal generated by your algorithm. Allow me to suggest a crude, yet effective starting place.

Suppose you are back watching an equity as it approaches what you think to be a $20.00 resistance level. You need a metric to identify the strength of the resistance, and have one of the given order books displayed above in Figure 2. In the order book on the right, you could find the average price it would take a market buyer to pass four levels of depth. This calculation ((2*19.99+8*20+1*20.01+2*20.22)/12) shows resistance strength of 20.0008.

Using an analogous calculation on the left shows resistance strength of 20.0063, a greater value that can act as a metric defining a resistance level.

The more expensive it is to surpass a level of resistance, the less likely it will happen.

Exactly how this metric is created has many degrees of freedom. If you suspect a resistance level exists at $20.00, you could initiate a position that depends on how expensive a set of market orders would have to be to consume past the resistance. You could also alter how far deep to look when calculating the average price.

These two measures involve simple math, and provide a deeper insight to market movements. They are based on the absolute lowest levels of price formation by supply and demand, and are certainly items to consider when building a full system. In my next post, I will provide a more specific strategy to consider implementing.

Limit Orders at Best Bid Best Offer

by 2 Comments

Earlier this year I was testing market making ideas in detail. To quickly recap, the idea of making a market entails placing limit orders either at the current best bid and best offer (BBBO) or further away. When the limit order gets filled, the trader achieves a major advantage. Spread costs no longer weigh on the returns.

A failed trading experiment with MB Trading to earn commissions segued into a trading project that I ran on behalf of a client. The key issue was that the strategy could only profit if trading costs were kept to an absolute minimum. The easiest way to reduce cost is to not pay the spread.

Creating a limit order at the best bid/best offer accomplishes that. Whenever a buy limit placed at the current bid price receives a fill, the trader can immediately sell the trade at the bid free of charge.

That’s the theory, anyway. The BBBO idea makes the assumption that another bid or offer would continue to stand behind my own at the same price. Like most trading experiments, that assumption flopped. The NinjaTrader strategy would post the order, then leave it at the original price without changing. Sometimes the order sat for 10-20 minutes. It was only when the price crossed a custom indicator line in the opposite direction that the strategy cancelled the order and placed a new one at the best bid/best offer (BBBO).

The most informative insight that I learned about the experience was the fill rate on the orders. Curiously, placing them at BBBO only resulted in execution 75-80% of time on several hundred attempted trades.  MB Trading’s ECN program is rather small. I chalked the low execution rate up to a lack of price takers, even on a major pair like the EURUSD.

So, we switched to Interactive Brokers and tried again. I was quite surprised to find the same result; orders placed at BBBO only filled 75-80% of the time. The really profitable trades always got skipped over as the market found its stride and zoomed upwards.

I don’t know how much of this falls in line with normal market mechanics and what responsibility high frequency algorithms might play in the low execution. It appears commonly accepted that HFT algos routinely engage in games where false orders are displayed with the intention of lifting the market prices, only to slam them back down again when a sucker accepts the lifted price. An interview with a high frequency trader made me wonder if perhaps the HFT algos were lifting the price hoping that my strategy would be their sucker. An interview from two months ago added to my lingering suspicion.

A second example: HFTs can model other traders’ behavior. When someone trades through Scottrade or Interactive Brokers, their order has a unique number attached to it – the same number every time a client places an order. This number is bundled with all relevant trade information (time, price, etc.) and sold as an encrypted “enhanced data feed.” An HFT can then use those past results to predict the trader’s behavior.

I don’t believe this happens at MB Trading, although my order sizes were admittedly so small it’s easy to see individuals among the order stack. I routinely look at the market depth and feel like I can identify the orders of individual retail traders.

I can’t help but wonder how many people experience a surprisingly low fill rate using limit orders at BBBO. Use the comment section below to share any relevant stories that you may have.

Grid Strategy

by 4 Comments

Grid strategies are common alternatives for traders that do not have an opinion on market direction. They are almost exclusively associated with forex trading. I’ve never seen grid trades in any other context.

The goal of a grid strategy is to outline a ranging or trending bias without committing to the underlying direction. That may sound confusing.

The goal is to only summarize the type of market. Trending conditions prevail in today’s market. If a trader did not know the future direction of the price, he might place stop entry orders a certain distance away from the current market. If the market happens to increase 10 pips, then perhaps that triggers a buy stop order on the expectation of a continuation. Another 10 pips later, another buy order triggers, and so on. The goal is to keep stacking orders on one way moves.

Ranging grids work on the opposite assumption. They use a limit entry instead of a stop entry order. The grid assumes that if the price drifts very far, then it’s likely to come back to where it started.

Problems with Grid Strategies

Position sizing and money management are always some of the biggest concerns with an expert advisor. Two of the more common approaches that I see in grids are either to use a fixed lot size or to use a Martingale approach.

I see merit in the idea of varying lot sizes at different levels. Martingale, however, takes it way too far. It’s a mathematical fact that it will blow up at some point in time. A more reasonable approach is to increase at a very slow rate like 10% as trades become increasingly likely to exit. If a trade is decreasingly likely to exit, the idea of not trading should come to mind. Alternatively, trading smaller sizes is always an option.

The other problem is that grids only work at the moment in time where it’s applied. When a ranging grid expert advisor is placed at the top of a range, the grid will correctly anticipate the market conditions but poorly implement the prediction. The top of the range means that the price falls back down into the middle. The grid, however, assumes it was placed in the middle. The grid buys as the price falls into the mid-range on the errant expectation that it will return to the top of the range.

This is precisely what I dislike about grids. They are totally blind to the context of their current placement. They are best used, in my opinion, in the context of slight directional bias but where outright trades may not make sense.