Timothy Lewkow, Author at Algorithmic and Mechanical Forex Strategies

Order book Slope

October 2, 2013 by Timothy Lewkow Leave a Comment

I used to make levered bets in the option market during earnings season. Pick a direction, choose an option near the money and close to expiration, then hope. It seemed just as stupid as buying a plane ticket to Las Vegas and betting a black chip on red. By the time all expenses were weighed, I was always charged a premium for entertainment.

I have since learned a new orderbook measure that could provide an edge. It’s an idea that has been around for some time, but has plenty of room for optimization. The math is somewhat cumbersome, but I though I would take a few posts and explain why I might cancel my future trips to Vegas.

Limit Order book Information

When collecting data for an algorithmic trading system, the two most common pieces of information are quoted price, and volume. That said, it would be silly to think that information contained in the limit order book contains no excess details toward the price formation process. Not only do you know the current price, but you also know the current price that others are demanding! Creating a solid model of the order book is the key to understanding aggregate liquidity and trading interests of market participants.

Research in order books has found a significant relationship between volume, volatility, and a value known as order book slope. The ideas are somewhat nontrivial on the surface, so I want to summarize the key points and mathematics for easy implementation.

Order book Slope

If you Google around, the correlations between volume and volatility (typically denoted as the “Volume-Volatility Relation”) are used for several liquidity measures in the market. The Volume-Volatility Relation has empirical findings in stocks, bonds, currencies, and futures, though I will stick to equities in this post for demonstration.

To apply Volume-Volatility to order books requires a notion of slope in a limit order book. Consider the following example.

Two different types of stocks show very different slopes in their order books. The idea of a line connecting the dots intuitively explains the idea of order book slope.

In a market frozen in time, each circle in this example represents a value in the limit order book waiting to be consumed by a market order. In a simple equity market, each tick on the x-axis represents one penny movement in the stock price, while the y-axis represents the total aggregate volume on each side of the market.

The main observation you should be making is the shape of the volume in the market for these two hypothetical securities. The new IT firm seems to be more linear in creating a v-shape, while the blue chip firm has more curvature as it flattens out at the top.

The intuition is there too. Really, take a hard look at Figure 1 and think about it. Blue chip companies like Apple and Google have great volume and perhaps a tighter consensus of price during a typical day of trading. Therefore, it is reasonable to think that just about 100% of the limit order book volume occurs within 5 ticks of the current market price.

On the other hand, think about those penny stock start up companies we have all been enticed to buy at some point of our trading career. Companies with unknown valuation and future potential will have limit order book volume spread ten or more ticks from the current market price. Some people think these companies are doomed to tank as a product of price inflation, while others may own the product produced and realize the potential as early investors. The two diverse opinions will have valuations that come out drastically different and causing the plots above.

Aggregate Orderbook Slope

Now consider what happens over time. If you collect data over time and take an average, the result will appear as in Figure 2.

Order book slopes published by Naes and Skjeltorp

Here, about 50% of volume in the blue chip firm has limit prices within 5 ticks (left) while only 10% of volume is within 5 ticks for the young IT firm on the right.

The published research finds a negative relationship between (average) order book slope and the variation of analysts’ earnings forecasts. This means that as the slope becomes more linear (right plot of figure 2) investors disagree more about the value of the firm and thus higher volatility will likely result.

Creating a solid model of the order book is the key to understanding information about aggregate liquidity and trading interests of market participants.

In fact, the order book slope has been shown to have relation to volatility, volume, and the correlations between volume and volatility– all three of which could be profited on.

Surprisingly, in the mid 1990’s, people were studying the shapes of these curves and relating them to the different liquidity providers present in the market. The shapes were shown to find the probability of informed traders completing market orders at any given moment! See the Glosten model for more info.

Making a precise definition of order book slope requires a blog post of explanation of itself. The math looks daunting at first glance, but as I will explain later, has great intuition, elegance and simplicity. Along with that, the formulas are tangible and should leave you with a relatively simple programming problem (assuming you have the data)

Finding Free Stock Data

September 24, 2013 by Timothy Lewkow 2 Comments

I spent the past two years writing an algorithmic trading system that ran in my bedroom while I left the house, worked my day job, and lived my life. The idea took several months to derive, implement, and perfect while overcoming more obstacles that I could have imagined. By far, the largest problem I have encountered is finding good, free stock data.

Stock Tick Data

The smallest increment of free data I could find is on a Norwegian website that can be read in English by searching “NetFonds” in Google and selecting “Translate this page.” Along with currency and commodity data, NetFonds also has tick data for all NASDAQ, NYSE, and AMEX stocks. Getting the data for free is easy, but takes some effort to understand.

Getting free stock data is as refreshing as a day at the beach.

To start, enter the following URL into your browser

http://hopey.netfonds.no/tradedump.php?date=[date]&paper=[stock]&csv_format=txt

Here, there are two parameters that you need to alter.

[Date] – Should be replace with a date in the form YYYYMMDD, so for example 20130919 would be the data obtained from Thursday, September 19, 2013. In my experience, data goes back around 15 days, but I can’t guarantee this for every stock. Generally, I take and store yesterday’s data today.

[Stock] – This is where you replace the name of the ticker to collect. The catch is that you must know the exchange code.

NYSE code is ‘N’ — for example, to collect Macy’s, [stock] = M.N
NASDAQ code is ‘O’ — for example, to collect Google, [stock] = GOOG.O
AMEX code is ‘A’ — for example, you get the picture

The data displayed has a time, price, and quantity in .txt format. Everything looks self explanatory, expect for the time column. I elaborate by example of the first entry I see.

time = 20130919T153000

Translated as 2013, 09 (sept), 19 (day), Time, 15:30:00

which seems weird, but remember, you are collecting data from a Norwegian website and Oslo is six hours ahead of New York City time. Considering military time format, 15:30:00 is really 3:30 in Norway, which is 9:30 in EST and the market open. Notice that under this logic, the last data point during open market hours is represented by the string

time = 20130919T220000

By far, the largest problem I have encountered is finding good free data

You will also notice that some rows have identical time stamps. This should be interpreted chronologically with the logic that the price is changing several times per second. Recall how price changes.

Finally, I want to note that all times outside 15:30 and 22:00 are after hours transactions. You can always see after market activity on Google finance. Try searching for Apple, and check “Extended Hours” under the settings link under the given chart. Grey prices are transactions that occurred after hours.

Order Book Tick Data

The best free tick order book data I could find displays only the best bid and ask quotes for a given time. Nevertheless, there are endless ways this information could be used to improve a system.

Again on NetFonds, try pasting the following URL into your browser:

http://hopey.netfonds.no/posdump.php?date=[date]&paper=[stock]&csv_format=txt

with the same date and stock convention used above. Notice you have a few extra columns corresponding to volume and best bid/ask in the market.

For this data set, you will see that extended hours quotes extend far more than in tick data, though the spread widens considerably. Extended hours trading is considered risky due to this lack of liquidity, but this is a topic of it’s own.

Google Finance Data

Data can be found here, and follows very similar conventions to NetFonds though data comes in every minute. The URL is

http://www.google.com/finance/getprices?i=[PERIOD]&p=[DAYS]d&f=d,o,h,l,c,v&df=cpct&q=[TICKER]

[PERIOD] – Time interval in seconds
[DAYS] – Historical data period. For example [DAYS] = 10d asks for the last ten days
[TICKER] – The stock symbol. No codes necessary, so AAPL works just fine

Yahoo Finance Data

Similar to Google Finance and Yahoo, the general URL is given by

http://chartapi.finance.yahoo.com/instrument/1.0/[TICKER]/chartdata;type=quote;range=1d/csv

Frequency is seconds, and historical range available is 5 days.

Obtaining the Data

Programming languages have an age old trade off. If you want a fast language, you have to sacrifice in learning non-trivial languages and concepts. If you want a code that downloads the above data sets, and you want it to work tomorrow, you have to sacrifice in using a slower language.

For me, Mathematica and Python were extremely intuitive to use on day one, and both have built in functions to browse and download data. I also learned to use Apple Script on my mac with very little effort. This was nice because I could program my computer to wake up in the morning, go to a website, and download the latest data.

The speed trade off from not using a language like C++ was assumed away for me. Unless you pay top dollar, you have to assume that the data you are downloading is somewhat perturbed and there is nothing you can do about it.

Support and Resistance Metric

September 17, 2013 by Timothy Lewkow 2 Comments

Converting back of the envelope ideas to trading algorithms is extremely challenging, but requires significant attention. This could be the single item that makes your system great. In this post, I suggest a way to find more defined support and resistance metric using limit order books.

Creating a Order Book Metric

To start, I want to create a frozen moment in time where two order books appear as in the following picture. For the sake of this example, I assume other signals have detected a $20.00 resistance level.

Support Resistance Metric Limit Order Book Comparison

Figure 1: Comparison of two limit order books during a suspected $20.00 resistance level.

Blue represents the limit bid and orange the limit ask. The order book on the left suggests strong resistance at 20, while the order book on the right suggests weakness . If you’re not convinced, see my post on the support and resistance in the limit order book.

The idea above is great in theory. How can you quantify the above two situations so that it’s actually useful? Otherwise, it’s just another fluff technical analysis piece with no real substance.

A successful method applied to Figure 1 should determined if a short position should be entered at $20.00 (left) or not (right).

The first of two decisions that you must make is how far to look into the order book — a value I will call N. The above two order books have N = 4 because each side of the book contains orders at 4 different price levels.

Of course this will depend on your access to data. However, you might also find through back testing that increasing N provides no useful information.

The next decision you have to make is how to weigh the volume in the order book.

You might be foolish to think that each level of the order book is created equally.

For simplicity, I am going to describe a linear weighting system where ticks closer to the price midpoint are given more weight. Referring to the first order book example, N=4 implies we need four weights on the ask side. My system has weights that satisfy the following conditions:

w1 – corresponds to $19.99
w2 – corresponds to $20.00
w3 – corresponds to $20.01
w4 – corresponds to $20.02
w1+w2+w3+w4 = 1
w1 > w2 > w3 > w4

These weights essentially alter the volume in the order book so that overcoming the level of resistance is a more pronounced event. Here’s how you can choose them for a linear case.

w1 = N = 4
w2 = (N-1) = 3
w3 = (N-2) = 2
w4 = (N-3) = 1
total = w1+w2 + w3 + w4 = 10

Through a process of normalization, the final weights are found as follows

w1 = N/total = 4/10 = .4 = 40%
w2 = (N-1)/total = 3/10 = .3 = 30%
w3 = (N-2)/total = 2/10 = .2 = 20%
w4 = (N-3)/total = 1/10 = .1 = 10%

Notice that the whole system comes by simply defining N, thus is easy to generalize and back test in your trading system.

The next step is to compute the average weighted price that would have to be paid if a market buyers chooses to pass the level of resistance. Again, I will refer to figure 1 on the ask side of the market. This will show the difference between a strong level of $20.00 resistance (left ask book) and a weak level of resistance (right ask book)

Figure 1 repeated

Average price (left ask book) : $19.99*2+$20.00*8+$20.01*4+$20.02*5/19 = 20.0063

Average weighted price (left ask book) : $19.99*2*w1+$20.00*8*w2+$20.01*4*w3+$20.02*5*w4/(19) = 20.0022

Average price (right ask book) : $19.99*2+$20.00*8+$20.01*1+$20.02*1/12 = 20.0008

Average weighted price (right ask book) : $19.99*2*w1+$20.00*8*w2+$20.01*1*w3+$20.02*1*w4/(12) = 19.9989

From the weighting system, you can see that a assumed resistance of $20.00 is passed by only paying an average (weighted) price per share of $19.998 during a time of weak resistance.

The weighted average price shows more defined support and resistance levels

Generating Signals

Finally, to generate a solid metric, I considered the percent movement a stock would have to take to move past a weighted average price. For example:

Left ask book (strong resistance) : 100*(current price – average weighted price)/current price = 0.11%

Right ask book (weak resistance) : 100*(current price – average weighted price)/current price = 0.094%

My rule is set so that if the current price must move over 0.10% weighted average order book price (for a given value of N) then the resistance level is strong. Less that 0.10% makes me think the resistance is weak, and would signal for me to not initiate a short position.

Support and Resistance

September 11, 2013 by Timothy Lewkow Leave a Comment

The Limit Order Book is lurking behind every price tick in every market you can imagine. From the ill-liquid real estate market, all the way to high frequency bond trading, the limit order book determines all price movements.

A simple example in my last post of supply and demand demonstrated how price changes in an equity market. I made several arguments justifying the existence of a bid ask spread, and showed how this leads to price formation. My goal in this post is to find clarity in the foggy world of support and resistance using limit order books. Support and resistance information can be used to build confidence when entering or exiting a trade.

Imaginary Price Levels

If you try searching for support and resistance, a wealth of information can be found usually in the form of some article accompanied by several charts with lines claiming to have found the magical levels.

Every such chart I have found, however, has one single thing in common. The stock price always, at some point, just slightly crosses these horizontal lines. To me, it feels like a slap in the face… the ultimate I told you so from market experts that can apparently see into the future. Here’s a great example I picked up on Google.

Figure 1: Support and Resistance levels breached leaving question of imaginary lines.

I realize that mathematical definitions of these levels exist, and I realize that human psychological traits are often correctly considered. Nevertheless, they still lack precision! The arguments for true black and white support and resistance levels always must always have a fair amount of uncertainty. Sadly, this is just another part of working in this business.

If I see a price move past a resistance level in real time (i.e., not being able to see the entire future nicely displayed in front of me like Figure 1), I often question if the level has been breached. This could perhaps leave you in the worst possible position as the market rockets the wrong way. The limit order book can reduce this uncertainty by displaying real information.

If you try searching for support and resistance, a wealth of information can be found usually in the form of some article accompanied by several charts with lines claiming to have found the magical levels.

Suppose a stock is testing a human psychological resistance level of $20.00 and your algorithm has signaled that you initiate a short position. You wish to enter the market at the highest possible price without missing the peak. More importantly though, you wish to confirm a resistance level still exists . If the current order book is displayed as below left, you would have confidence that enough sell pressure is present to hold the resistance.

Figure 2: The left side of the image shows more market depth on the offer (orange), which is resistance. The right image shows light depth, which is the absence of resistance.

On the other hand, if the order book is displayed as above right, it it would take only a moderate collection of market order buyers to break the $20.00 level– and break it fast in this example. Short sellers would run to cover, and the market could swiftly move against you.

Measure Support and Resistance

I found some research out of Wharton suggesting an order book metric (cumulative depth), and have heard more advanced ideas shared in my personal research symposiums. That said, I think this situation is being made too complex.

Translating the above example into math should be straight forward, and customizable to the strength of signal generated by your algorithm. Allow me to suggest a crude, yet effective starting place.

Suppose you are back watching an equity as it approaches what you think to be a $20.00 resistance level. You need a metric to identify the strength of the resistance, and have one of the given order books displayed above in Figure 2. In the order book on the right, you could find the average price it would take a market buyer to pass four levels of depth. This calculation ((2*19.99+8*20+1*20.01+2*20.22)/12) shows resistance strength of 20.0008.

Using an analogous calculation on the left shows resistance strength of 20.0063, a greater value that can act as a metric defining a resistance level.

The more expensive it is to surpass a level of resistance, the less likely it will happen.

Exactly how this metric is created has many degrees of freedom. If you suspect a resistance level exists at $20.00, you could initiate a position that depends on how expensive a set of market orders would have to be to consume past the resistance. You could also alter how far deep to look when calculating the average price.

These two measures involve simple math, and provide a deeper insight to market movements. They are based on the absolute lowest levels of price formation by supply and demand, and are certainly items to consider when building a full system. In my next post, I will provide a more specific strategy to consider implementing.

Limit Order Book

August 28, 2013 by Timothy Lewkow 3 Comments

I remember the first time that I really sat down and thought about it. Why exactly does a stock price change? Shrinking the economy and the number of shares helped. Examples starting with 10 oranges together with supply and demand arguments sparked good ideas. But, expanding a simple scenario into a full blown economy with high volumes and different order types never made any sense.

The story is not complete without considering the information contained in a limit order book. It’s the absolute best source for highlighting buying and selling power in a market in real time. The information within the data often results in more desirable entry and exits points.

A simple example of a limit order book

The orange squares represent units of stock that you can buy at market

Suppose that each block represents one share of stock on both the bid and ask side of the market frozen in time. The volume of shares in the above plot are limit orders waiting for execution or cancellation.

Say that Frank comes along and wants to buy 5 shares using a market order. In that case, his order will be filled immediately.

Remarkably, the current quote displayed of $20,26 is not where Frank can trade- there are no shares available at that price. The quoting convention reflects the spread rather than tradeable prices.

The 5 empty colored squares represents the 5 shares that Frank bought with his market order

The 5 empty colored squares on the right represents the 5 shares that Frank bought with his market order

The order is filled by sellers in a first in, first out (FIFO) process. Those who waited the longest in the order book receive the first execution.

Frank’s market order for 5 shares receives execution at two different prices. The first 2 shares fill at $20.27. The depth of market at that price is only 2 shares, forcing him to sweep the $20.27 price and move on to the next available price at $20.28.

4 total shares are available at $20.28. Because Frank only needs an extra 3 shares, he completes his total order at this level.

The best offer displayed when Frank placed the trade was $20.27, but his average fill is $27.276 (2 * $20.27 + 3 * $20.28). The slightly worse price doesn’t have anything do with slimy brokers. Slippage is the natural result of buying more shares than there are shares available.

Try making Frank a more aggressive buyer. Say he wants nine shares. Large orders receive worse fills because they suck up most of the liquidity on one side of the market.

Why A Spread Exists

Before answering this question, it is first worth understanding the difference between a quote driven and an order driven market.

Order driven market:

• Displays all of the current bids and asks across the market

• Has complete transparency

Quote driven market:

• Displays bid and ask prices from market makers, dealers, or specialists.This is the norm among retail forex brokers

• Often provides a guarantee that an order is filled

A quote driven market has more moving parts and will likely be involved in any market you wish to trade in. Therefore, it is a good idea to think about the existence of a spread in this setting.

When you post an order in a quote driven market, the dealer will either fill it with their own inventory or match you with another market participant. For this reason, part of your transaction cost goes to the dealer who has done this work for you.

In a simple model, the bid ask spread is the price that aggressive traders must pay to have their order immediately filled– think buying and selling the same security at almost the same instant. The spread is the compensation to a dealer for offering that immediacy.

A good way to think about the size of the spread is to consider a market with several competing dealers. In this case, there are two primary scenarios:

1. If the spread is too high, more dealers will enter the market to gain profit from the large bid/ask spread

2. If the spread is too low, dealers will lose money, and exit the market

These two factors ensure that the liquid market dealers make normal profits, and that spreads are of reasonable size.

Supply and Demand

The existence of a spread is quite natural and leads back to the simplicity of supply and demand. Start the argument small and work your way up! I found a great example in an article by Glenn Curtis on August 19 with the following story.

Suppose that a one-of-a-kind diamond is found in the remote countryside of Africa by a miner. An investor hears about the find, phones the miner and offers to buy the diamond for $1 million. The miner says she wants a day or two to think about it. In the interim, newspapers and other investors come forward and show their interest. With other investors apparently interested in the diamond, the miner holds out for $1.1 million and rejects the $1 mil- lion offer. Now suppose two more potential buyers make themselves known and submit bids for $1.2 million and $1.3 million dollars, respectively. The new asking price of that diamond is going to go up. The following day, a miner in Asia uncovers 10 more diamonds exactly like the one found by the miner in Africa. As a result, both the price and demand for the African diamond will drop precipitously because of the sudden abundance of the once- rare diamond.

Imagine the diamond becomes more popular. More buyers want it. More mines open, and more sellers emerge. In a rational setting, this creates a quote driven a bid ask spread. Add enough volume, and before long, you are back to the first example.