Algorithmic and Mechanical Forex Strategies | OneStepRemoved

Become an Algo trader in Baby Steps

by 10 Comments

The world of traders is divided into two groups; those who trade using algorithms and those who don’t. Those who trade using algorithms, aka algotraders, are well aware of the advantages of trading with an algo… And for those who don’t use algos? They are equally aware of the algo advantages but are reluctant to dive into its complexity; they are deterred from learning how. But the road to success in trading is not by avoiding challenges, but overcoming them, perhaps in baby steps.

First Baby Step to becoming an Algo trader

Let me ask you a question. What do you think is the first thing you’d need to do to become an algotrader? You might respond, “Learn programming, of course! How else?” Well, you’d be wrong.

If you plan to learn programming for the sole purpose of becoming an algotrader, you’re likely to get lost. Eventually, in despair or frustration, you’ll give up on algotrading.

There are numerous languages, from MQL to R to Python, and you have to decide which one to start with. You might find yourself wasting valuable time learning far too many trivial functions. It may take quite a long time before you even create a trading algo, let alone a profitable one.

But there’s another way, which I call reverse engineering.

The first step is to figure out which trading algo you want to make. In other words, what are the functions and strategy it should implement? Then you test it and finally, move on to the programming part. This way, you’re focused like a spotlight. You know exactly what your algo should do and can focus on the exact functionalities you need to learn to make it happen.

Everything will just come intuitively; which language, how and in what. All the pieces will fall much quicker into place because you already know what to look for.

The best way to start is by using a flowchart. It is actually one of the first things you learn in programming schools.

And what determines the flow? Of course, it’s the conditions. What we call the “ifs.” “Ifs” can be one condition or have many “ands” and “ors.” That’s how you decide what your algo should do in any given circumstances. The conditions to the algo are what the brain is to the body; they do the thinking.

Second Baby Step: Use Excel

Once you’ve made a flowchart of conditions, rather than using a complex tool, use Microsoft Excel or some other spreadsheet software.

Algo

Source: MT4

Extract historical data of a price data; start by using only the closing price. If you want to trade on a daily interval, extract daily data and so on. Use a separate column for each of the conditions in the flowchart, that way it’s easy to write and figure out. Add another column for a buy and sell output to mark when you “opened” and when you “closed” a position. And finally add one column of accumulated profit/ loss.

Use separate columns for each conditions and fill the chart. All you need now is to run a linear chart on one column, accumulated profit/loss, and look at the algo’s historical performance.

Once you master the basics you can move on to move advanced back-testing and curve fitting. But for a start that will do.

Algo

Finally, Learn to Program

Ok, you’re not a “baby” anymore; you have a good idea of what an algo trader needs to do. Now that you’ve got the concept down pat, you’re ready to begin learning programming.

In terms of which language to start with, that will depend on the circumstances. If you are already using an MT4 platform, it’s a no brainer; learn MQL which runs on MT4. The MQL website is filled with “how to” materials. And since you already have a diagram of your algo and an idea what you want to do, finding your way should be simple.

But if you trade with a different broker, you’ll have to decide which best suits. Personally, I recommend starting with MQL, just because it’s easier. Then, you can move into Python, which is a rather easy language to learn or C++ if your algo needs to work fast. If you algo is heavy on the math then perhaps R would fit.

Here are some places you can learn online:

MQL

Coursera

Code Academy

EDX

Or, of course, you can learn from a book. I am more of a book person, but that’s just me.

Then there is another issue—API. API is the mechanism that enables your algo to communicate with your broker and execute your trades. Some brokers work better with certain programming languages than others. Most large brokers have communities and forums that can go into detail as to the best way to use an API.

Unlike the first two baby steps, no one would ever say that learning to program was a baby step. Rather, it’s more of a giant leap. It takes time to learn and to master, though it is worth it in the long run. Meanwhile, you can always use libraries of codes on the web for more complex algos. The thing is that once you’ve accomplished the first two baby steps, making the final leap into programming is a no brainer.

Resetting my demo testing

by 6 Comments

I mentioned in my new strategy post that live demo testing would last for two weeks unless bugs popped up. Bugs did pop up… but only in my rush to get ready for Thanksgiving!

I sat in my office frantically trying to wrap up code before business hours ran out on the day before Thanksgiving. The plan was to unplug for the long weekend. And, I did. Emails went on autoresponder and, aside from one night binging on Netflix, I didn’t see a screen for 4 days.

Rushing while problem solving, and especially programming, almost never ends well. I would never push untested code changes onto on a live account. But since I figured the change was simple and it’s only a demo account, it’s not a big deal if I mess it up. There went 4 days of live market testing down the drain.

The bug

I’m testing my live demo on FXCM US. As most US traders are aware, the FIFO rules imposed on FX brokers can get pretty annoying. I noticed that a small handful of my orders were being rejected in a special circumstance.

Say that I’m in a long trade and place my take profit at 1.0550. Now say that the signal is so strong that not only should I exit long, but I should also go short at 1.0550.

Anyone in a normal market would place a limit order to sell double the open position at 1.0550. That would cause the long trade to exit and a new trade to open in the opposite direction.

FIFO rules prohibit this because I’m already long. My intended bug fix was to place a take profit order to exit. The order to go short would then remain hidden in the code until the long trade exits. Doing the order logic this way follows my original intent, but also complies with FIFO.

The buggy bug fix

Using my long and then going short example, my long sets a take profit and then the code was intended to send 1 limit order to go short. I forgot to tell the code that it had already sent that one order. Instead, the code sent duplicate orders on every single tick.

Position sizes of 1 microlot mushroomed into several standard lots. Needless to say, the profit and loss began to swing wildly. The massive jump in size also swamped the statistics that I was hoping to measure with the data gathered so far.

It’s easier to start a new demo account, which is what I’ve done as of 5 minutes ago. One upside to using a new demo account is that I can run the code on an account size that matches my intended live testing balance. I plan to test with $2,000 instead of the $50,000 in the first demo.

The first few days of trading

equity curve

This was the first 3 days of trading

The first few days continued much in the same vein as in my original post. I’m looking forward to collecting data from the new demo account. You can expect to stay up to date on my progress if you’re a subscriber to my free newsletter.

How I picked my brokers

Using limit orders is extremely price sensitive. If you place a limit order to buy EURUSD at 1.0550, the price must exactly hit 1.0550 in order for that trade to execute.

One danger of trading on marked up spreads is that you don’t know whether the broker is widening the bid or the ask. If the interbank market quotes a 0.3 pip spread on EURUSD at 1.05480 x 1.05483 and the broker usually charges a 1.3 pip spread, then the broker can mark up the spread in several different ways.

  • Add the 1 pip markup entirely to the ask. The price on your screen appears as 1.05480 x 1.05493
  • Add the 1 pip markup entirely to the bid. The price on your screen appears as 1.05470 x 1.05483
  • Split the 1 pip markup across the bid and the ask. The price on your screen might be as 1.05475 x 1.05488

I of course have no idea which way the broker marks up the spread. If they’re smart, they will price shade in order to earn extra income from the easy order flow. There’s always the risk that the markup could cause one of my orders to not get filled, even though the real interbank market actually touched that price.

I’m doing my demo test at FXCM for three reasons.

  1. Commission only trading. It’s a pure interbank feed, so I don’t have to worry about how spread markups affect my execution.
  2. The commissions are very fair for a retail trader. It works out to $60 per side per million, which is only double the commission that a small institutional trader would pay. Retail rading costs have fallen dramatically in the past few years.
  3. Seer is already plugged into FXCM

When it comes to trade live, I’m planning to trade a personal account at FXCM US and a second account denominated in euros for my Irish company, Dominari (the same one that owns QB Pro). Dominari’s trading will go through Pepperstone, which offers the same commission only setup but 1) charges even lower commissions in euros and 2) offers much higher leverage.

Expert Advisor Assumptions

by 3 Comments

Many of our clients are ordering their first custom programming projects. They know the general framework of the expert advisor that they hope to create, but they are not sure of what to ask for specifically. The goal of this post is to outline how an EA functions in relation to the most commonly asked questions.

How do I select the currency pair and time frame that my expert advisor should use for trading?

An Expert Advisor runs on the chart where the user applies it. When you wish to trade an EA on the EURUSD M15 chart, open a EURUSD chart. Change the time frame to M15. Apply the EA. All decisions will be based on the chart.

What are inputs?

Inputs are variables that the user can change without additional programming. Traders often know the indicators that they like using. The settings are often less clear. Inputs allow for tweaking or experimenting without needing to email the programmer every time that you change your mind. A more concrete example is to consider an EA that uses a moving average. Is a 50 period MA better than a 55 period MA? Making the period an input allows the trader to experiment quickly and easily.

How does my EA choose how many lots it should trade?

I assume that you wish to trade fixed lot sizes unless you tell me otherwise. Most of our SOWs include a Lots input. Any trading signals that appear would enter with the amount that you enter for Lots. Clients frequently request other types of money management. You need to explicitly state the type of money management that you wish to include or else it will not be in the expert advisor.

Why does every SOW include a stop and take profit?

Mostly because it does no harm being in there. Setting them to zero will disable each of their functionalities. The vast majority of clients want the ability to select stops and limits. Our programming templates automatically include them to help speed up the development process and to reduce costs for our clients.

What is the difference between a generic trailing stop and a breakeven trailing stop?

A generic trailing stop is what most people think of when the word trailing stop appears. It maintains a set distance from the most favorable price seen. A breakeven trailing stop is more complicated. It was covered in a previous blog post.

What is the purpose of an SOW? Why are you so insistent on pinning down all of the details?

All of our projects are pre-paid. Before we required SOWs for all projects, the expectations between OSR and the client often differed. The customer expected one thing; we expected another. It’s worth taking the extra time to make sure that everyone understands the project and what is expected.

The SOW also allows us to develop a working relationship before any money changes hands. You feel more comfortable with the purchase when you already know that we understand the project from top to bottom.

I want to trade 10 currency pairs at a time. Why can’t a single EA manage all 10 currency pairs?

A single expert advisor technically could manage multiple time frames and multiple currencies. I recognize that the idea is nice; why manage 10 EAs when you could only manage 1 EA? The problem is that MT4 EAs depend on incoming ticks to update. If the expert advisor wants to trade the AUDUSD but it’s applied on the EURUSD, the trade won’t fire off until the EURUSD receives an incoming quote. That’s not good, especially with short time frames. There’s the risk of trades firing off later than expected, trailing stops taking too long to update, etc. More importantly, the execution will always bottleneck because of the trade context is busy error.

I welcome questions on these blog posts. If you’ve always wondered how EAs work and would like your question answered on the blog, then send me an email.

System Expectations

by Leave a Comment

I found myself outside today looking for an excuse to make a video. It’s 80°F / 27°C, sunny skies and a soft breeze in Dallas. The last place anyone wants to be is in front of the computer when the weather is this nice.

No doubt that some active daytraders or people that hate their jobs are thinking the same thing. I suspect that the motivation for most people making automated expert advisors is the dream of making money without doing anything. Turn on the software and wait for the trading profits to roll in. That was certainly the case with the company Forex Made Sleazy… I mean, Forex Made Easy several years ago.

We do have a handful of customers that trade profitably, but even then, it takes a long time for an automated system to get to the point where it’s largely hands off. The best conceived ideas, which I would define as plausibly worthy of my own investment funds, takes a bare minimum of several months to execute from start to finish. This also presumes the unlikely notion that the idea has genuine potential to start with.

Even the most simple, valid concepts encounter substantial setbacks before the system can truly run hands-free. It’s usually not some kind of epic programming disaster where the client wants black and the programmer makes white. Don’t get me wrong; communication is critical. The smoothest projects are always the ones where both parties understand one another readily.

Nonetheless, even the most well-oiled team experiences countless hiccups in the process of morphing from idea to reality. Simple ideas often fall the most vulnerable to real world problems. Trade execution stands out as the most common obstacle. If anything goes remotely unexpected, a potentially profitable scenario may lead to unexpected losses.

I worked with one client that came up with a simple idea that mathematically showed a heavy positive expectation. Yet when we launched the idea in the real world, the prices that the system absolutely required in order to function never came through. Slippage occurred precisely when it was the most damaging.

We had to go back to the drawing board looking for ways to re-engineer the expert advisor where the importance of execution declined. That setback alone took several months to overcome in any meaningful sense.

The take away here is that it’s totally unreasonable to expect to hire a forex programmer and expect a dramatic shift in profits and life style. The best ideas take several months before they are worthy of running their full account balance. Unfortunately, most of the ideas out there are not good to begin with. That’s why making an EA that is profitable over the long run is so incredibly difficult.

Develop Trading Strategy

by Leave a Comment

One common element unites 99% of the trading strategies that we’ve developed. The final product does not match the initial design.

All programming projects proceed in one of two ways. You can either develop a project from the top-down, or you can build from the bottom up. The former option represents the old way of thinking, especially within the university system. When professors assign a project, the requirements never change. The assignment remains static.

Real life rarely cooperates like that. Instead, human beings change their minds. They see the work in progress and decide that a minor tweak might dramatically alter the outcome. The expert advisor programmer is forced to constantly adjust to the moving goal posts.

I always tell clients that they will change their minds about some of the rules during the process. Most insist that they will not. They do, after all, expect them to work.

Then, the rules inevitably change for most of them. Theory goes out the window. The traders discover the real issues that expert advisors face in the market.

Programmers developed a new methodology to accommodate the constantly shifting rules. It’s called agile development. This problem doesn’t only occur with forex traders. It also happens with businesses making web sites, database programmers storing information, and really any project that involves human beings. People are prone to changing their minds. It’s our nature. Agile development is the programmer’s acknowledgement that getting upset at people for changing their minds is not entirely reasonable.

The agile approach breaks a project into time periods with a ranked list of objectives. The team sorts out which objectives are most important and do-able within the allotted time. The period starts and everyone rushes to accomplish as much programming as possible. Everyone regroups after 2-6 weeks to evaluate the progress and to deliver the updated version to the client.

Most expert advisor projects only last a few weeks, so a precise copy of the agile approach would not fit very well with developing a trading strategy. We flip the strategy development approach on its head by assigning ranked priorities with a deadline of 1-2 business days. The goal is to facilitate communication, which is inevitably the most challenging obstacle in this field. We also make the process less formal to speed things along.

The quick turnarounds lead to a series of yes/no answers. The client either gets excited that it works properly, or more likely, feels that we did not understand him correctly. The honest goal of this approach is to get the user away from explaining the idea for the 1,000th time in the same, identical way. The last explanation did not help the project move along. Client feedback from working software leads the programmer into understanding the concept on his own without making the client feel like he’s repeating himself.

I like to think of it as the client steering the driver’s wheel while the programmer controls the gas pedal. The strategy development process only works when the steering and power move in synch.

Overcoming expert advisor programming challenges

When clients feel like an expert advisor has not made the desired progress, the first step is to assign goals one at a time. Too often, we receive reports with 20 different bugs. The emphasis on priority falls away as the desire to “fix everything and fix it now” drowns out the details of each request. I always like to take a step back and reaffirm our commitment to developing the strategy, but we can only handle so many issues at a time.

I have a very patient client in New Jersey (I never though I’d say “patient” and “New Jersey” in the same sentence”) who ordered a lot of custom elements in a basic expert advisor. Most of the complications arose from the fact that I had to remotely access his computer with LogMeIn to do the programming. His indicator only has one license. Purchasing a second for the EA programming was not economical.

The requirements to remotely access the computer and the amount of custom code turned a simple project into a complicated one. The way I addressed the issue was to remove almost all of the custom components and to replace them with code from our usual template. Now that the client sees that the “strategy” stuff works, he found it much easier to write a checklist of bugs to fix. More importantly, seeing critical components of the code working reinforced his confidence in the company’s programming ability.

Although he remained patient throughout the debugging process, I could hear the flagging confidence in his voice before we switched gears. Changing the direction of the project showed him that the EA actually does work; it’s the little pieces and how they fit together that are the problem. Of equal importance was the fact that he felt confident in my ability to deliver the results that he wants.

The client gets full credit for releasing control of the debugging process, even when he wasn’t entirely comfortable doing so. The idea of papering over the custom steps with pre-programmed template code struck him as a step backwards. He nonetheless followed the programmer’s lead, letting him drive the process with himself providing feedback where it was needed.

Both parties agree that we’re once again heading in the right direction. The client sees obvious progress in his expert advisor. The programmer is able to manage the debugging process in a methodical, organized manner. Everyone is happy.

Realistic strategy development

The most realistic way to develop a strategy is to acknowledge that you don’t exactly know where you’re going. You know it’s likely to involve a certain indicator and that it either trends or ranges. The best way to prepare for the journey is to acknowledge in advance that it is indeed a journey. You more than likely will make changes, most of which you cannot expect.

It’s 100% certain that something is going to go wrong when you program your first project. It’s your strategy, but it’s really a two-man team that builds it. Make sure to pick someone that knows what do when problems come up.

Before you choose your programming partner, make sure it’s someone that you actually trust to get the job done. You ideally want to find someone that offers strong communication skills and appreciates the importance of a timely reply. Organizing the project and overcoming unexpected challenges helps everyone get back to their true passion of trading.