Algorithmic and Mechanical Forex Strategies | OneStepRemoved

Using Dow Theory in Forex

by 2 Comments

There’s hardly a trader, whether short term or long term, who doesn’t rely, in some form or another, on technical analysis. Yet many don’t know that the backstory behind what we today refer to as technical analysis is actually a collection of ideas on trading stocks. Some of those ideas are, in fact, more than a hundred years old; they are referred to collectively as Dow Theory. Going into the source, the so-called genesis of technical analysis can provide valuable lessons for a trader, even today and even in Forex. In this article I will go into some basic concepts of Dow Theory and offer my personal takeaways from it.

First, though, the backstory. The Dow Theory is named after Charles Dow, a financial journalist and one of the founders of the world-renowned Wall Street Journal. Dow had written a series of articles on his theories and concepts on market behavior, pricing and patterns for the Wall Street Journal. Those ideas were later developed further, refined and enhanced by followers such as William P. Hamilton, Robert Rhea and Richard Russell. The collection of ideas known as the Dow Theory therefor, encompasses concepts from Dow and his followers.

Dow Theory Concepts

With that brief history of the Dow Theory behind us, it’s time to get into the meat and focus on some key concepts and how they could be implemented when trading Forex.

I like to think of the Dow Theory as having two pillars—one theoretical, the other practical. The theoretical concepts focus on how to approach the market, the so-called theory behind trading.

The practical ideas are focused on things such as rising tops and bottoms, which confirm a bullish trend, and/or rising volumes, which confirms a trend’s strength. Since most practical ideas belong to basic technical analysis such as stretching a trend line, I will focus on the theoretical side which is often overlooked by traders.

Combining the two pillars should give the trader the proper approach and the necessary tools to beat the market.

Concept

The Dow theoretical side focuses on the benefits of the bigger picture. In other words, it focuses on what the broad market is doing rather than a specific asset or security or, in our case, a specific pair. Why?

Over the longer term, the broad market cannot be manipulated by any one player. It is true that over short durations, the broad market can be manipulated, but unlike a stock or a security, over the long haul there is no one factor with enough liquidity to manipulate the long term trend. That means that in order to profit one must first gauge the long term trend of the broad market, and only then can one make a decision on the next trade.

Moreover, according to the Dow Theory, the broad market prices all the knowns and even the potential unknowns that have a higher probability of occurring. In other words, the broad market is so big and diversified that the current trend and price behavior prices all the known positive and negative information as well as all that market participants believe could happen.

Dow in Forex

Although the Dow Theory was initially designed for analyzing stocks, the concepts stated above can provide some powerful insights into the Forex markets. If we refine the two concepts we get one clear idea; that is the best way to predict markets is to focus on the big picture and that means focusing on the long term and focusing on the broad market rather than a signal pair.

That means that focusing on the long term trend over months, and even years, can yield much better results than focusing on shorter durations. Moreover, in a more practical sense, longer duration charts tend to be easier to analyze with support, resistance and trends much easier to define. Personally, it is one of the key reasons that I prefer long term trades. Of course, short term traders can be highly lucrative and successful as well, but focusing on the long term trend is essential for crafting a strategy that could work over the long run. It allows you to identify areas of high volatility, areas where a pair is destined to have resistance, or areas where short term support can be broken.

Another important takeaway is focusing on the broad market. How does that come to play in Forex? It means that you should always aspire to analyze the big trend. In practical terms, it means, for example, that you should always analyze the Dollar Index before trading a dollar pair, to figure out the long term trend on the dollar. It also means that, if you trade a low liquidity pair such as the USD/BRL, you should first identify the overall trend in the FX market, risk on or risk off.

As seen in the sample below, the EURUSD is trending higher which means a bearish Dollar. But on the other hand in the second chart , the Dollar Index which represents the Dollar against a basket of currencies, is trending higher as well suggesting the exact opposite, a bullish dollar. Since Dollar index represents the big picture for the Dollar it is the one we should relate to when determining the long term trend.

Dow Theory

Dow Theory

The Bottom Line

Of course, there are many more takeaways and more layers to the Dow Theory and it is always a good idea to go over the original books and learn from the source, whether it’s the articles by Charles Dow himself or The Stock Market Barometer written by William P. Hamilton and The Dow Theory by Robert Rhea. But, as an experienced trader, through the years I have found that the best takeaway from the Dow Theory is that its emphasis on the big picture improves your chances to avoid manipulation and areas of unexpected market reactions which, in turn, makes a strategy more successful. It doesn’t mean that you have to trade only over the long term but it does mean that you have to first figure out the long term before anything else.

 

Import Stock Symbols to NinjaTrader

by Leave a Comment

Importing numerous stock symbols into NinjaTrader is a tedious process. If you’ve ever used the Instrument Manager and want to create a list of the most liquid ETFs, you know what I’m talking about.

There is a far, far easier way to import multiple stock symbols into NinjaTrader without using the Instrument Manager.

Go to the control center, the main part of the NinjaTrader Program. Click on File, Utilities then Import Stock Symbol List.

import stock symbol NinjaTraderThe new screen brings up a text area. Type in the list of symbols or click on the small Load button in the middle of the screen towards the left side. That allows you to import a list of tickers if they’re already typed into a file.

Import stock symbol screen in NinjaTrader

“Traded on” is the exchange where these instruments trade. You can use Nyse if you’re not planning to trade live. The symbol mapping is only important when it’s time to use NinjaTrader for live order execution.

My final tip is that you can assign all of these instruments to a list. The screenshot depicts a few commodity ETFs. It makes a lot of sense to categorize those instruments into an ETF list. You could click the “New” button to the right or, if you already created a list with that name, select it from the drop down menu.

Could An Overnight Edge Enhance Your System?

by Leave a Comment

Markets are capable of changing dramatically while we are asleep. This can be seen in all types of markets for many different reasons.

Many traders avoid ever carrying a position overnight. Others ignore this concept because they believe that the impact is minimal over the long-term. Some traders simply avoid taking positions ahead of key news announcements, which is usually what trigger big overnight moves.

overnight edge

Sometimes markets make significant moves while they are closed for the night. Could we develop a system to capitalize on these overnight edges?

Jeff from Alpha Interface took a different approach to this idea when he wrote a recent post summarizing some research he found concerning large overnight price moves in US stocks. Based on the research Jeff wrote about, there may be an overnight edge in trading these moves.

The research Jeff summarized was a paper by Berkman, Koch, Tuttle, and Zhang. They examined 3,000 US stocks from 1996 through 2008 looking for strong positive overnight moves. Here is what they found according to Jeff:

They found a strong tendency for positive returns during the overnight period followed by reversals during the trading day.

This behavior was driven initially by an opening price that was high relative to intraday prices.

It was concentrated among stocks that had recently attracted the attention of retail investors (typically due to a news announcement), it was more pronounced for stocks that were difficult to value and costly to arbitrage, and it was greater during periods of high overall retail investor sentiment.

So basically, when stocks make large positive moves overnight, they tend to go a bit too far. Jeff’s piece continues my suggesting how traders could take advantage of this:

Their tests were predictive and thus indicated that postponing purchases of these stocks from the open until later in the day can avoid these hidden costs.

Similarly, selling these stocks at the open, rather than later in the day, can lead to major improvements in performance.

Obviously, this idea would need a lot more development before it could ever be a legitimate strategy. However, it is a very interesting concept.

It would be interesting to research how many “significant” overnight moves actually happen over the course of a year. This would probably vary a great deal based on the stock universe and your definition of “significant.” From there, you would want to determine the percentage of those trades that would have been profitable and calculate the returns on both the winners and losers.

While this system would require a lot of effort to develop, it could be the type of short term strategy that keeps your capital out of the market most of the time. That would give it the added benefit of protection from Black Swan events and some slight interest income from the risk-free asset you kept the capital in when there were no trades to be made.

Retail trader disadvantage

by Leave a Comment

Michael Halls-Moore invited a reply to one of my tweets last week, “Retail traders have an advantage over the pros? Me thinks not.” He wrote a great overview of why the institutional traders look longingly at the retail crowd and all the hoops that they don’t have to jump through.

His points are all valid, but he overlooked the big picture. Pricing is everything to a trader. Retail traders get the short end of the stick when it comes to the cost of doing business.

The cost of trading is massively disproportionate

Let’s say that you’re a would be quantitative trader and that you’re looking for opportunities. Let’s say you trade mini lots in the forex market with 60% accuracy and 1:1 risk reward ratios. If you’re not familiar with what a typical trading system looks like, those numbers means that you have an enormous edge.

Some of the less reputable forex brokers out there charge 3 pip fixed spreads. If you’re trading with a broker offering fixed spreads, I urge you to start price shopping. Fixed spreads are wildly overrated. You pay a huge premium for the certainty of a fixed spread. I can’t think of anything remotely plausible to justify them.

The larger forex brokers charge typical spreads in the neighborhood of 2 pips on the largest majors. Although most seem to find this reasonable, the comparison between a 2 pip average spread and institutional spreads is night and day.

Do you know what the average EURUSD spread looks like on the interbank market? It’s often 0.2-0.5 pips. Retail traders pay an average markup of over 300% on their trades.

retail trader pricing

Retail traders facing the institutions is a bit like David and Goliath.

Retail forex prices have declined in recent years. A few brokers like MB Trading and Pepperstone offer raw spreads with commissions tied to the dollar volume traded. These are, in my opinion, are about the fairest prices available to low balance traders running an expert advisor.

The best deal available to semi-institutional forex traders (CTAs, large balance retail traders, etc) is Interactive Brokers. The customer support is famously poor; they’re cheap for a reason. IB also offers raw spreads with a commission.

My experience with IB has been excellent, but you need to trade size for the economics to work. A 0.5 pip typical spread is great, but the 2 mini-lot minimum trade size and $2.50 minimum commission really adds up. Trading with IB doesn’t approach institutional type pricing until your average trade size approaches 1 standard lot.

So, how does pricing affect the final outcome with our 1:1 risk reward strategy that wins 60%?

  • Free trading: After 100 trades, you’ve earned $600 and lost $400. The hypothetical net profit is $200.
  • Fixed spread: You’ve spent $300 in spread costs to enter 100 trades. The total net profit is -$100 ($200-$300).
  • Average retail: You’ve spent $200. There is no profit because you breakeven ($200 hypothetical profit – $200 in costs). However, your broker loves you for doing that many trades.
  • Good retail pricing: Let’s say the average cost of a trade is 1.3 pips after commissions. You’ve spent ~$130 placing 100 trades. The total profit is $70.

Even with good strategies, the profitability of your algorithm is as simple as choosing the cheapest broker.

Equities pricing

Trading stocks is even more expensive than forex. I remember back in the day when I thought Scottrade was cheap for offering $7 commissions. It gets worse and worse when you go through the list of stock brokers. Most of them try to get away with charging $7-10 per trade. If customer service is important to you, then those are the shops to look at.

If your top priority is trading profitably, then again, broker selection is critical. The only way that a small guy can make it is by chipping away at the costs. Interactive brokers is again a great option, charging fractions of a penny per share traded. If you decide to trade 2 shares of Google (GOOG: $1,017 per share) or 1,000 shares of Fannie Mae (FNMA: $2.35 per share), the transaction costs are tiny. Two ticks in your favor is all it takes to cover the trade.

You might be thinking that I said two ticks in forex is expensive. How can I say that two ticks in equities is reasonable?

Volatility. Two ticks in the stock market is a little hiccup. It’s not at all uncommon to see highly liquid stocks move 2-3% in a single day. Forex is only interesting because of the leverage. The currency pairs themselves rarely move more than 1%, and that’s usually on major news.

Risk Management

Every employee knows that they’re only one mistake away from getting fired. That’s the reason that everyone hates having a boss. There’s a single person with unilateral authority to financially murder you. Who’s going to look upon that as a good thing?

Well, the truth is that bosses exist for a reason. It’s someone that calls you out when you do something stupid. More importantly, the boss has the power and influence to ensure that you stop doing stupid things.

The dream of entrepreneurship is not having a boss. You go on vacation when you can, you don’t have to play office politics, you don’t have to waste time selling good ideas. You just go out and do them.

Even with good strategies, the profitability of your algorithm is as simple as choosing the cheapest broker

I can tell you as a small business owner that the negatives stand out strongly in my mind. When you don’t have someone to hold you accountable, even if it’s a mentor, you make many more dumb mistakes than you should. It takes incredible discipline to hold the line consistently. Knowing that I’m not going to look stupid or have to explain myself to anyone probably gives me a lot more false confidence than I really need.

Self-employed traders working at home experience the same thing. Who calls you out when you’re trading just because you’re bored?

The decline in the trading account points out the obvious, but that’s not enough to necessarily stop the bad behavior. We’re social creatures. Most people need to speak with other people to maintain their sanity. When you’re trading at home alone, it takes a lot of effort to ensure that you’re getting enough social contact. A good boss prevents you from indulging in bad behaviors.

Conclusion

Selecting the right broker is enough to determine whether or not a good strategy will wind up making money or not. It’s expensive to trade. The bigger you are, the better your pricing.

Retail trading prices have reached a point where it’s at least possible to trade profitably. Nonetheless, the number of strategy types out there is limited because the lower, shorter term strategies are prohibitively expensive to trade.

The quantitative traders and hedge funds get the more active strategy space to themselves. Their trading costs are so low that they’re really the only people that can afford to trade actively.

SPY Crisis Strategy

by 3 Comments

Yesterday’s musings on an S&P 500 doji strategy led to a general discussion of stocks and market crises. I promised to analyze a price-moving average cross strategy and to analyze the performance in times of exceptional volatility.

The results are in and they’re exactly what I predicted. I’m shamelessly tooting my own horn on this one – it’s so rare where strategies do exactly what I predicts.

SPY Crisis Strategy Returns

The direction of the returns matches any traders definition of crisis and regular trading periods over the past decade

SPY Crisis Strategy Rules

The trading rules only initiate short trades. No long positions are allowed.

Enter short next bar at market when:
The price crossed and closed below the 20 day SMA on the last closed bar
The trader believes that a crisis environment either currently exists or is about to exist

Exit an open short trade when:
The price crosses and closes above the 20 day SMA on the last closed bar

The position size is equal to a fixed dollar value divided by the current share price. As an example, SPY currently trades at $169.24. If you wanted to control a position size worth $1,000, then the number of shares is the floor of $1,000/$169.24 = 5 shares.

This strategy is intended to be timely for the current trading environment. Based on all of my proposed definitions below, most of the crisis alarm bells are ringing at the moment.

Defining a crisis

The most difficult part of this type of strategy comes from defining a “crisis environment” quantitatively. Crises don’t happen very often by definition, so I don’t think it’s a worthwhile endeavor to try to quantity the crisis bit. That said, a few obvious crisis indicators come to mind based on basic market mechanics.

PE Ratio

The morons on Tout TV (CNBC and company) keep on screaming how cheap stocks are. I’m not a fundamental trader, but the PE ratio contains useful information. Even the most hard core technical analysis buff would agree that companies generating huge positive cash flow and earning growth have to appreciate at some point. The argument isn’t about if that type of stock will rise; it’s just a question of when.

I don’t see how anyone could possibly look at the current PE ratio of 19.3 and argue that stocks are cheap. They aren’t. Stocks are currently very expensive based on a historical comparison.

VIX

VIX is a CBOE benchmark index that allows traders to compare the price of front month options traded on the S&P 500. A more detailed explanation of the VIX is available on Wikipedia if the concept is new. There’s nothing magical about the 20 level. It’s my general experience that most traders consider that number the one to watch. They think of VIX < 20 as "normal" and VIX > 20 as a severe market move.

VIX danger level

Most traders regard a VIX above 20 as a dangerous level.

Put-Call Ratio

Options are effectively leveraged bets on market movements with fixed downside risk. When traders load up on puts, they’re expecting the market to fall. When traders buy more calls, they’re expecting the market to rise.

The put call ratio is simply the number of put contracts traded / the number of call contracts traded. A number > 1 means that more puts were purchased that day than calls, indicating an expectation of a market drop.

Theory has it that short term traders are wrong, making the put call ratio a contrarian indicator. I see the put call ratio as more of a lagging indicator.

ES Put call ratio

When a move is real and already happened, traders react too late and buy protection that they no longer need. The 2008 financial crisis a great example when the ratio spiked to 1.5, a wild number. Just in the past week the ratio went as high as 1.3 before settling back down. The volatility in the number indicates a panicky crowd in my opinion.

Margin debt

Leverage is a two way sword. The theory is that it’s a way to multiply returns by risking debt in the market.

Most traders, and especially retail traders, wind up using leverage as the rope to hang themselves with. Stocks are most commonly purchased with cash among investors. Unlike forex and futures where almost every trader enters a position with leverage, the average retail stock trader enters a position using only the cash present in his account.

An increased willingness among traders to move from cash to margin debt is typically a sign of froth, bubble fever or whatever you want to call it. The chart of margin debt from Business Insider and Zero Hedge show that stocks are currently trading near historical highs.

margin debt business insider

Margin debt Zero Hedge

Conclusion

The 20 day SMA price cross strategy is a great way to run account protection whenever market warning signs are going off. The warning signs may not predict the precise market turning point, but the strategy can function as an effective form of insurance.

The strategy would roughly break even over time if someone were foolish enough to run it that way. Say that you mistime the crisis. Big deal. This type of strategy can run for months without causing irreparable harm to the account.

The signals can run in the background. If you’re only a little bit right with your crisis predictions, the risk reward ratio is massively in your favor. If you’re wrong, the consequences appear to be slow losses that lose a couple of percentage points per quarter. If you’re feeling edgy, I think it’s a great strategy to run in the background to calm your mind.

Finding Free Stock Data

by 2 Comments

I spent the past two years writing an algorithmic trading system that ran in my bedroom while I left the house, worked my day job, and lived my life. The idea took several months to derive, implement, and perfect while overcoming more obstacles that I could have imagined. By far, the largest problem I have encountered is finding good, free stock data.

Stock Tick Data

The smallest increment of free data I could find is on a Norwegian website that can be read in English by searching “NetFonds” in Google and selecting “Translate this page.” Along with currency and commodity data, NetFonds also has tick data for all NASDAQ, NYSE, and AMEX stocks. Getting the data for free is easy, but takes some effort to understand.

free stock data

Getting free stock data is as refreshing as a day at the beach.

To start, enter the following URL into your browser

http://hopey.netfonds.no/tradedump.php?date=[date]&paper=[stock]&csv_format=txt

Here, there are two parameters that you need to alter.

[Date] – Should be replace with a date in the form YYYYMMDD, so for example 20130919 would be the data obtained from Thursday, September 19, 2013. In my experience, data goes back around 15 days, but I can’t guarantee this for every stock. Generally, I take and store yesterday’s data today.

[Stock] – This is where you replace the name of the ticker to collect. The catch is that you must know the exchange code.

NYSE code is ‘N’ — for example, to collect Macy’s, [stock] = M.N
NASDAQ code is ‘O’ — for example, to collect Google, [stock] = GOOG.O
AMEX code is ‘A’ — for example, you get the picture

The data displayed has a time, price, and quantity in .txt format. Everything looks self explanatory, expect for the time column. I elaborate by example of the first entry I see.

time = 20130919T153000

Translated as 2013, 09 (sept), 19 (day), Time, 15:30:00

which seems weird, but remember, you are collecting data from a Norwegian website and Oslo is six hours ahead of New York City time. Considering military time format, 15:30:00 is really 3:30 in Norway, which is 9:30 in EST and the market open. Notice that under this logic, the last data point during open market hours is represented by the string

time = 20130919T220000

By far, the largest problem I have encountered is finding good free data

You will also notice that some rows have identical time stamps. This should be interpreted chronologically with the logic that the price is changing several times per second. Recall how price changes.

Finally, I want to note that all times outside 15:30 and 22:00 are after hours transactions. You can always see after market activity on Google finance. Try searching for Apple, and check “Extended Hours” under the settings link under the given chart. Grey prices are transactions that occurred after hours.

Order Book Tick Data

The best free tick order book data I could find displays only the best bid and ask quotes for a given time. Nevertheless, there are endless ways this information could be used to improve a system.

Again on NetFonds, try pasting the following URL into your browser:

http://hopey.netfonds.no/posdump.php?date=[date]&paper=[stock]&csv_format=txt

with the same date and stock convention used above. Notice you have a few extra columns corresponding to volume and best bid/ask in the market.

For this data set, you will see that extended hours quotes extend far more than in tick data, though the spread widens considerably. Extended hours trading is considered risky due to this lack of liquidity, but this is a topic of it’s own.

Google Finance Data

Data can be found here, and follows very similar conventions to NetFonds though data comes in every minute. The URL is

http://www.google.com/finance/getprices?i=[PERIOD]&p=[DAYS]d&f=d,o,h,l,c,v&df=cpct&q=[TICKER]

[PERIOD] – Time interval in seconds
[DAYS] – Historical data period. For example [DAYS] = 10d asks for the last ten days
[TICKER] – The stock symbol. No codes necessary, so AAPL works just fine

Yahoo Finance Data

Similar to Google Finance and Yahoo, the general URL is given by

http://chartapi.finance.yahoo.com/instrument/1.0/[TICKER]/chartdata;type=quote;range=1d/csv

Frequency is seconds, and historical range available is 5 days.

Obtaining the Data

Programming languages have an age old trade off. If you want a fast language, you have to sacrifice in learning non-trivial languages and concepts. If you want a code that downloads the above data sets, and you want it to work tomorrow, you have to sacrifice in using a slower language.

For me, Mathematica and Python were extremely intuitive to use on day one, and both have built in functions to browse and download data. I also learned to use Apple Script on my mac with very little effort. This was nice because I could program my computer to wake up in the morning, go to a website, and download the latest data.

The speed trade off from not using a language like C++ was assumed away for me. Unless you pay top dollar, you have to assume that the data you are downloading is somewhat perturbed and there is nothing you can do about it.