Test your trading ideas

Working 8 days a week

December 6, 2016 by Shaun Overton 22 Comments

Reaching an all-time high in my equity curve means it’s time to buckle down and keep improving. My Dominari strategy has done very well over the past 7 months and especially this and last month.

Is the party going to continue?

I certainly expect so. Drawdowns are inevitable, but that’s part of trading. Short-term performance is exciting, but my ambitious goal is to turn my starting balance of €8,000 into €50,000 within the next 3 years. As of this writing, I’m at €9,323.

You’re probably wondering how a 16% profit leads me to extrapolate an annual return of nearly 100%. The answer is that I dramatically changed my leverage at the end of September… just in time for that ugly drawdown. If I was trading on my current leverage, the current live return would be around 40% (i.e., right on track to hit my goal).

What really counts is what I’ve really done. So far, I’m up €1,323 with another €40,677 to go by December 6, 2019.

The research for Dominari is effectively finished. It’s been slightly more than a year since I began researching the strategy. Although minor variations of Dominari popped up or came from traders copying my signals, none of them improved the long term outcomes.

One version that improved the risk profile was to trade with limit orders. The original blog post mentioned limit orders, but the variation placed them considerably further from the current market than what I tried previously. I also lacked a system for choosing settings appropriate to every pair, which I’ve more than likely resolved. The problem is that I have a million things on my to-do list and only 8 hours a day. You’ll see some of my top projects when you scroll down.

Pilum: The latest and greatest

Pilum is a strategy based on a statistical process that identifies momentum. One of the scary elements about most quantitative strategies is that most of them are mean-reverting. They buy when the price drops and sell when the price rises. The approach is favorable (i.e., profitable) in the long run, but it takes some psychological fortitude to trade.

Pilum is a major advancement because now I’ll have a strategy that should profit exactly when Dominari is most vulnerable to a drawdown.

The new strategy uses limit orders to enter the market. Something like 90% of these orders never execute. But when they do execute, I win 75% of the time. Additionally, my profile of winners to losers is very comfortable.

Most traders understand the ideas even if the statistical jargon is unfamiliar. Pilum’s biggest winner is larger than its biggest loss. The average winner is bigger than the average loser. And, it wins 77% of the time.

So far, I’ve done a sort of piecemeal backtest using R. When I finish the Quantilator (see below), I’ll redo the backtest in a fully fledged trading platform. More than likely, I’ll use QuantConnect to test the strategy level approach.

Trading platforms drive me crazy! The biggest problem that I have as a trader is continuously reallocating capital across my portfolio. MetaTrader, NinjaTrader and the likes implicitly assume that I want to trade some percentage of my account balance on every trade. Either that, or that I trade fixed lots.

Trading that way is extremely inefficient. I’m trying to trade 40+ currencies, so I need to be able to decide which ones need the money for trading and which ones don’t have signals. Then, among the ones that do have signals, I need to dish out their allocations proportionately. The allocations aren’t the same for each instrument. If you know of any good FX platforms that meet this requirement, then let me know in the comments section.

Testing Pilum on its own is important. More important than the performance of Pilum is how that performance interacts with Dominari. That means taking the daily equity values of each currency. Does Dominari lose when Pilum wins and vice versa? Should I allocate capital 50-50 between the strategies or does one strategy deserve the lion’s share of the portfolio? Is one strategy so good that it should get all of the money?

The main concern with portfolio allocation is how it relates to leverage. Dominari historically make 96% annual returns, inclusive of trading costs. But, it’s also trading with leverage of roughly 19:1. It’s possible for markets to rip over stops and create significant losses.

USDCHF lost 40% of its value in one hour in January 2015. Say that the scenario was even more extreme and that nobody could place a trade during that time at any price. That 40% move is multiplied by the 19x leverage used. That’s a theoretical 800% loss; more than the money in the account.

Everyone loves leverage because they think of profits. Leveraging losses is a lot less exciting.

So, if you could earn 96% annual returns and only use 5:1 leverage, that is exponentially superior to earning 96% on 19:1 leverage. I need to compare the returns of Pilum to Dominari per unit of risk. That allows me to do cool things like

Minimize the negative variance of the returns
Increase absolute return
Dynamically increase/decrease strategy allocations if I find patterns in their interactions

It’s a lower tech way of averaging strategies, like the litte guy’s version of what Numerai is doing… except that I have to create all of the strategies myself.

Quantilator

I spent the last few months sending surveys to segments of my subscribers asking how I can better serve you. Articles about indicators are overwhelmingly my most popular content. The trouble with that content is that I can’t complete the research fast enough to keep up.

The most valuable thing I’ve learned from the developing algorithms for the past 11 years is my process of deciding whether or not an indicator offers predictive value.

Say that you’re interested in gaps: do gaps predict future returns? What I normally do is code a gap indicator in R, implement my analysis methodology and give a verdict. My methodology is kind of like a system for building systems. Using my approach usually takes an hour for every new idea that comes along.

An hour is pretty short. An hour is REALLY short compared to when I didn’t have a research methodology. I used to waste months on junk strategies.

With Quantilator, I’ll be able to analyze anything in under 5 minutes. I’ll only have to follow 3 steps:

Run a script in MT4 to export price data and indicator data
Upload the exported data to Quantilator
Analyze the results

This tool will be 100% free. I’m planning to go through the most popular indicators in MetaTrader to analyze whether or not they offer an edge. I’m building a library of small edges that can be combined into powerful strategies like Dominari and Pilum. And, in the spirit of open source, I plan to make that library available to you for free.

I’m writing this tool in Django, which is a Python framework for displaying web content. The initial version will do the analysis. I’m hoping to use this as an education tool. A bit of momentum can justify make it a fully fledged library with sample data, indicators and training videos and more.

Quantilator’s name comes from a key concept in my system analysis methodology; I break data into quantiles. These quantiles calculate average market returns for a given period of time. The quant in Quantilator refers to quantiles, but I really like the implied double entendre of making you a quant. After all, that is what I’m doing for you!

Update: The Quantilator is now freely available at http://q.onestepremoved.com/

Analyze your Trading Algo with 3D Charts

June 28, 2016 by Lior Alkalay Leave a Comment

These days, any mention of the term 3D is associated with entertainment. But in fact, when it comes to charting, and more specifically to charting your trading algo, 3D charting is not only insightful but provides important practical advantages.

The most common chart to measure a trading algo is profit over time. That lets you know how much money the algo is making over a specific duration, usually from a few weeks to several months. As the chart below illustrates, it gives you a good idea how well your trading algo performs over time and it gives an indication of the periods when it was underperforming.

The thing is that, while profit over time are the two most important dimensions, they leave plenty of dimensions out—dimensions that can help you answer important questions. Such as why, during a specific period, was your trading algo under-performing? Or how much risk are you taking in a given time? Often, the answers to such questions can be the difference between profit and loss, between success and failure of a strategy.

Trading Algo in 3D

First things first. Before we start running 3D charts on our algo it’s important to go over a few practicalities and make the 3D chart work for you.

Assuming you’ve already exported the data of your algo Profit and Loss to Excel you’re likely to have two columns of data, e.g. Time and Profit. Adding a third column will allow you to run a 3D chart, whether it’s volatility, risk or whatever additional dimension you deem necessary.

Once you have your three columns you click to generate a chart—you must choose a type called 3D surface chart. As you will notice, almost always, the Time stamp will be the X-Axis, Profit the Y-Axis and our third parameter will be the Z-Axis.

Now comes the important part—making the chart comfortable to work for us.You must remember that our goal in using a 3D chart in the first place was to identify areas of either exceptional profits or exceptional losses to optimize our algo.

As can be seen in the charts below, Excel divides the Y axes into ranges and each range is colored. The best practice is to choose the same color for levels that are not exceptional and select a contrasting color for the highest range and another for the lowest range. This allows us to spot the exceptional.

The Z axes changes the angle of the chart; the steeper the angle, the higher our Z parameter—say risk or whatever else we choose.

And finally, make the 3D chart clearer through formatting the Plot area. Play with the Y rotation angle as well as the Debt Level until you are comfortable working with the chart

Trading Algo Case Studies

Once you are clear as to how to make a 3D chart, it’s time to decide which dimension is relevant. Usually, besides time and profit, the following dimensions are worth considering—risk, volatility and duration.

For example, the chart above shows a profit over time of a specific strategy; let’s call it Strategy A. Suddenly, out of the blue, the profit plunges rapidly. It’s not clear why, yet.

Then we add another dimension—risk. Risk, in this case, will be the Dollar amount risked in a given moment. Now, the reason is apparent; just before the profit collapsed, risk was rising, as well. Maybe leverage jumped, maybe several positions were opened simultaneously; it depends on the strategy. But by using a 3D chart we were able to easily detect where trouble was coming from.

Using 3D charts is not only good to spot weaknesses in a strategy but strengths. Let’s take a look at another strategy, which we’ll call Strategy B.

We will test how Strategy B performs during volatility. In this case, the volatility will be the standard deviation of each pair we trade. What we see is interesting. When volatility is high, Strategy B performs exceptionally well and not so well when volatility is average to low.

In such a case, we should consider using the strategy only during high volatility to optimize returns.

More uses could be to measure duration per trade. If the duration is getting longer at certain areas perhaps the trigger for the entry or exit is not working well. The benefit with using a 3D chart here is that we put the opening time stamp on the X-Axis and the closing time stamp on the Z-Axis so we can actually analyze duration per trade over time. A 3D chart then is much more accurate than a two dimensional chart where duration is a trailing average.

In Conclusion

There are endless samples and ways in which 3D charts can allow you improve your trading algo and identify both weaknesses and strengths within your strategy. Sure, you can manage with a 2-dimensional chart. But the benefit of 3D charting is that, many times, it allows you to zoom in and identify areas of change much easier.

Optimizing Your Algo: Tips for Beginners

June 6, 2016 by Lior Alkalay Leave a Comment

You have created a trading algo. The Algo is profitable in the backtester. Before unleashing it with real money, you’ve got to first tighten the screws. That is, ensure your algo is fine-tuned so it can deliver optimal returns. There’s one major challenge ahead of you.

Firstly, your strategy is rather simple. You may go through the detailed process of building and optimizing a strategy, including curve fitting, correlation and so on. But you want a simpler process; something leaner, that will fit your modest needs.

Secondly, you may not yet have mastered the full technique of optimizing. You are still learning and want to try off with a simple process.

One technique I find especially simple in optimizing your algo is probability. The probability method, in essence, contains many components of a full optimization technique. However, it tends to rely more on common sense and logic to narrow the options and optimise the algo. That makes it a pretty good way to begin the entire concept of optimizing a strategy. Moreover, you will find it easier to digest.

The essence of the strategy—optimize by elimination.

Algo Case Study

The following Algo is a simple one. Let’s call it RSIMV which is RSI and Moving Average. Here is what RSIMV describes through conditioning:

If Open Positions = 0 then

If (MA(30)>MA(14)) and RSI=<60 then

Open Buy (50,000) {It will buy 50 lots}

Set Stop Loss = Price – 50{Pips}

Set Limit= Price+ (50*2)

End

The strategy: If the moving average cross points on a bullish trend and the RSI is equal or below 60 it means that the rally has some length before reaching an oversold level (RSI above 80). That points to a good buying opportunity.

Looking at RSIMV, you can conclude there are four parameters to optimize: RSI and two Moving Averages

Starting with the Moving Averages, we will look at the 14 and the 30 days. Seemingly, the options are endless, with many combinations of moving averages to test. In theory, that is correct, but that is where probability comes in.

MT4

When we look at the chart, we can see that the longer the averages (orange and red) the lesser the chance that there is a combination of a low RSI and a bullish momentum.

Moreover, a combination of a low RSI and a bullish signal only occurs when the two averages, the fast and the slow, have more or less a 2 to 1 ratio (such as a 30 and 14).

Those two conclusions help us narrow the parameters we are looking for.

The highest likelihood of finding a better set of averages is with faster averages, not slower, and those that have a ratio of 2 to 1. And let’s not forget we already know that 14 and 30 works. So we shouldn’t move too far up the scale.

We will use 25 and 12 as the first combination and 20 and 10 for the second. Both are faster than the original parameters, have a roughly 2 to 1 ratio, and are close to the original settings.

25,12
20,10

Moving into the RSI parameter, narrowing the options is even simpler. We know the RSI cannot possibly be higher than 60, because then we will be left with insufficient upside before the pair turns oversold.

On the other hand, if we try an RSI below 40 it’s unlikely that it will occur while the moving average cross is bullish.

Since, as in the Moving Averages case we know the original setting worked, we know we only need a minor tweak. With no way to go but up we are left with two reliable options – RSI<55 and RSI<50.

Hence our options are:

C) (55)
D) (50)

As we can see from testing all the alternative parameters, what we needed was a better entry for the RSI. As we intended… minor tweaks.

Don’t Optimize Too Much

Ironic as it may sound, optimization sometimes has a downside. At times, we are tempted to over-optimize to such an extent that our newest strategy no longer resembles our original, pre-optimization plans. That can throw us into an eternal loop and waste precious time. Don’t be tempted! Don’t fall in love with the optimization process. After all, optimization is merely tightening the screws, not building the engine. If your strategy works, confine your optimization to minor tweaks. If it doesn’t, optimization won’t help and you’ll need to start from scratch.

And, finally, a practical tip; always keep records of the results of your original strategy and compare it to your current, post-optimization strategy. This way you can always make sure that you’ve really optimized your strategy.

The Bottom Line

Sure, the optimization technique isn’t perfect. But the take away here is that if you really understand your strategy, you can use logic to in order to find better settings. If you’re a beginner and lack the knowledge for advanced optimization techniques, optimizing through the logic of probability is a powerful tool to have.

Oscillators you need to use

May 31, 2016 by Lior Alkalay 8 Comments

Oscillators, one of the most interesting groupings of technical indicators, are designed to signal overbought and oversold levels. Oscillators are a family of indices that go beyond the mathematics. They focus on one important thing and that is momentum, or more specifically changing momentum. Before we delve into which Oscillators are best to use and how, let me save you some unnecessary pain. Let me tell you first what oscillators aren’t.

How Not to use Oscillators

Some traders believe that Oscillators are some sort of magic indexes. Rest assured when I tell you they are not. Oscillators’ main use is not to tell you whether to buy or sell. Rather, they alert you to when it might be a good time to execute a buy or sell strategy. That is a very big difference. Those who attempt to use Oscillators as an ultimate buy or sell signal should be ready to learn a tough lesson. Those that will use it to fine tune their timing, however, will find Oscillators a very powerful tool.

Now that we’ve established what oscillators are good for let’s focus on which oscillators are worth your time and how to use them.

MACD Indicator

Perhaps the most widely used Oscillator in Forex, the MACD needs no special introduction. What it does need is a proper explanation of how to use it and when.

The idea of MACD is to signal your entry point when you’ve already figured out where the trend is going. It’s not going to alert you to a trend. What that means is that you first have to perform your technical analysis. Once you reach a conclusion, then you can use the MACD.

On MT4, the MACD comes with default parameters (12, 26, 9). 12 represents the fast Exponential moving average, 26 the slow exponential moving average and 9 the Simple MACD average. Usually, when you trade on a daily basis, those parameters are fine.

Now in the chart below we see two points, A and B. In point A, the histogram moved above the average and that is supposed to be a buy signal. But, technical common sense says that a pronged bearish trend cannot end abruptly without some form of double bottom. Hence, one should ignore that signal.

But in point B, that’s a different story. After a double top that hits a resistance level and hits the trend average, there’s a case for a short. But we need to know when. Notice how after the second top the histogram in the MACD falls again below the average? That’s our mark and that is how you use MACD to time your trade. Once again, the lesson here is that Oscillators are for timing, not for point to the pair’s direction.

Stochastic Oscillator

This is one of the most interesting indicators in the Oscillators family. What I like about this indicator is that it essentially gives you a 2-dimensional picture of overbought, oversold and momentum. Unlike the MACD, that’s not always accurate on overbought/oversold level.

The idea of the stochastic oscillator is twofold. First, it’s normalized from 0 to 100, anything below 20 is oversold and anything above 80 is overbought.

Second, using the convergence between the %K line and the %D line tells you something. Not only can you tell when there is an overbought/oversold level but also when the trend turns bullish or bearish. Thus it affords a 2-dimensional use of momentum. Confused? Here’s a classic example of how I would use a stochastic oscillator.

In the first part, we can see that in point C, after the pair has bottomed, the stochastic oscillator was below 20. That signaled an oversold level. We can conclude that there the pair is bottomed out, through a double bottom pattern. We can use the oversold level as enforcement but wait before dipping our toes into a buy position. Only in point D, as the blue line crosses the red line, we get our signal for entry.

However, one other thing is important to note and that is point E. In point E we get the blue line crossing the red line as it does with C. However, since the cross occurs very close to the overbought signal that should deter us from establishing it as an entry. Which means if the crossing in point D was close to the stochastic 80 level, we should have avoided entry.

Average True Range

Last but not least, one of my favorite indicators, the Average True Range or ATR for short. Unlike the other two Oscillators the ATR is useful in anticipating potential rises or falls in volatility. Also unlike the other two, there is no oversold or overbought levels. If the ATR is high it suggests volatility is high. Conversely, if the ATR is low it suggests volatility is low.

How can we use this to predict volatility? We know, of course, that volatility is cyclical. Thus we can assume that when the ATR is at record lows for a prolonged period (point F) it’s a signal that a spike in volatility is coming. And, indeed, we can see volatility did come and we got the big spike.

If we decide to use the support as an entry signal we can ATR to gauge whether there’s a chance our buy trade will have a strong momentum. If the ATR was at a record high when we decide to buy that would have been a cautionary note. Not to entry but because it signals that volatility might fall and that means momentum might have been weak. The same principle, of course, works for a sell signal.

Beware of a Trading Strategy That Does Too Well

May 10, 2016 by Lior Alkalay 2 Comments

Can a trading strategy be doing too well? That sounds counter-intuitive, certainly. Like suggesting that someone can be too rich or too successful. You might be thinking there’s no such thing. But when it comes to managing your trading strategy, one that is performing too well is a warning sign. That’s because, quite often, an over-achieving trading strategy can turn into a honey trap.

In this article we will focus on a couple of cases studies where a “too good” strategy is something to watch out for. Hopefully, that’s a lesson one can learn from now rather than, painfully, at some later date.

A Trading Strategy with a High Win Ratio

The first case study is one of a strategy of mid- to short-term timeframes. In this case study, the frequency of executed trades is higher than 100 a month.

Usually, a solid strategy has an average win ratio of 45% to 60%. That ratio might sometimes be lower if the risk reward ratio is very strong. But higher than that is almost always is unsuitable.

In the strategy below we can see that the average win ratio per month runs from as low as 44 to as high as 64. Sometimes it is on the rise in the higher range and sometimes in the lower range. Regardless which, eventually it will all revolve around the average.

For example, in the month of January 2014, the win ratio was 52%. That means that for every 100 trades executed, 52 were profitable.

But in the second part(red), we can see things start to go a bit too well. The win ratio jumped to above 80% and stayed above that level for 3 months. Naturally, at 80%, that was far beyond the norm. What would come next was inevitable.

After 3 months of more than 80% of profitable trades, the win ratio fell to roughly 20% for the following 3 months. Because a long-term win ratio always has a long-term expectancy of maximal 50-60%. That means that a prolonged period of a high win ratio can be followed by a prolonged period of a very low win ratio.

Why is it Risky?

First, if you had 3 months of roughly 20% win ratio means you lost 80% of the trades. That means that, assuming you trade 100 times a month, you lost 80 trades. That’s a pretty hard hit. And if you risk $50 in every trade you lost $6,000 on aggregate (80 losing, 20 profitable) after 3 months. That is a big blow.

The second reason it is risky is more psychological but still very common. That is that many traders fail to realize that an overly high win ratio is very temporary in nature. Having failed to see that, what do they do? They raise their leverage. And when the sweet turns to bitter and they have only a 20% win ratio for 3 months, then what? All they’re left with is an empty account.

What should you do? This is rather simple; you reduce the risk you take per trade by lowering your leverage. It’s true you will gain less but you will also avoid the pitfall that comes after. Of course, if your leverage is already low and your account can sustain a prolonged period of a low win ratio than let statistics reign. But be prepared, psychologically, for choppy times.

Trading Strategy Profits that are Too Much Too Fast

The second case study is common in trading strategies that are low in frequency. The durations may range from a few days to even a few months. Let’s say you enter into a buy trade after your trading strategy indicated a bullish trend. You set the limit at 1,290. On average, you expect the limit to be reached within 3 to 4 days. But suddenly the trade advances in a fraction of the time and reaches relatively close to your trade.

Why is it Risky?

Because many times traders insist on waiting for their limit to be hit so they leave the trade open. But then the pair moves into overbought territory and before you know it the trend has reversed. You either close the position with a much smaller gain or, even worse, you lose the trade. Because you trade at a low frequency, this one outcome can be quite painful. You lost precious time when your position was open and missed the potential profit that you could have made.

What should you do? The solution here is rather simple, as well. You could use a trailing stop loss which is the most obvious tool to avert such cases.

Alternatively, you could use oscillators to identify a potentially hazardous situation. Add a component to your trading strategy to alert you if your pair reaches an overbought (or oversold) level. Make it a practice to exit the trade even if the target was not reached.

Become an Algo trader in Baby Steps

April 4, 2016 by Lior Alkalay 10 Comments

The world of traders is divided into two groups; those who trade using algorithms and those who don’t. Those who trade using algorithms, aka algotraders, are well aware of the advantages of trading with an algo… And for those who don’t use algos? They are equally aware of the algo advantages but are reluctant to dive into its complexity; they are deterred from learning how. But the road to success in trading is not by avoiding challenges, but overcoming them, perhaps in baby steps.

First Baby Step to becoming an Algo trader

Let me ask you a question. What do you think is the first thing you’d need to do to become an algotrader? You might respond, “Learn programming, of course! How else?” Well, you’d be wrong.

If you plan to learn programming for the sole purpose of becoming an algotrader, you’re likely to get lost. Eventually, in despair or frustration, you’ll give up on algotrading.

There are numerous languages, from MQL to R to Python, and you have to decide which one to start with. You might find yourself wasting valuable time learning far too many trivial functions. It may take quite a long time before you even create a trading algo, let alone a profitable one.

But there’s another way, which I call reverse engineering.

The first step is to figure out which trading algo you want to make. In other words, what are the functions and strategy it should implement? Then you test it and finally, move on to the programming part. This way, you’re focused like a spotlight. You know exactly what your algo should do and can focus on the exact functionalities you need to learn to make it happen.

Everything will just come intuitively; which language, how and in what. All the pieces will fall much quicker into place because you already know what to look for.

The best way to start is by using a flowchart. It is actually one of the first things you learn in programming schools.

And what determines the flow? Of course, it’s the conditions. What we call the “ifs.” “Ifs” can be one condition or have many “ands” and “ors.” That’s how you decide what your algo should do in any given circumstances. The conditions to the algo are what the brain is to the body; they do the thinking.

Second Baby Step: Use Excel

Once you’ve made a flowchart of conditions, rather than using a complex tool, use Microsoft Excel or some other spreadsheet software.

Source: MT4

Extract historical data of a price data; start by using only the closing price. If you want to trade on a daily interval, extract daily data and so on. Use a separate column for each of the conditions in the flowchart, that way it’s easy to write and figure out. Add another column for a buy and sell output to mark when you “opened” and when you “closed” a position. And finally add one column of accumulated profit/ loss.

Use separate columns for each conditions and fill the chart. All you need now is to run a linear chart on one column, accumulated profit/loss, and look at the algo’s historical performance.

Once you master the basics you can move on to move advanced back-testing and curve fitting. But for a start that will do.

Finally, Learn to Program

Ok, you’re not a “baby” anymore; you have a good idea of what an algo trader needs to do. Now that you’ve got the concept down pat, you’re ready to begin learning programming.

In terms of which language to start with, that will depend on the circumstances. If you are already using an MT4 platform, it’s a no brainer; learn MQL which runs on MT4. The MQL website is filled with “how to” materials. And since you already have a diagram of your algo and an idea what you want to do, finding your way should be simple.

But if you trade with a different broker, you’ll have to decide which best suits. Personally, I recommend starting with MQL, just because it’s easier. Then, you can move into Python, which is a rather easy language to learn or C++ if your algo needs to work fast. If you algo is heavy on the math then perhaps R would fit.

Here are some places you can learn online:

Or, of course, you can learn from a book. I am more of a book person, but that’s just me.

Then there is another issue—API. API is the mechanism that enables your algo to communicate with your broker and execute your trades. Some brokers work better with certain programming languages than others. Most large brokers have communities and forums that can go into detail as to the best way to use an API.

Unlike the first two baby steps, no one would ever say that learning to program was a baby step. Rather, it’s more of a giant leap. It takes time to learn and to master, though it is worth it in the long run. Meanwhile, you can always use libraries of codes on the web for more complex algos. The thing is that once you’ve accomplished the first two baby steps, making the final leap into programming is a no brainer.

Big change to Dominari

March 9, 2016 by Shaun Overton 24 Comments

I said it here and here and here. The biggest issue with my Dominari is trading costs. Things aren’t going to really take off until I do one of two things.

Reduce the trading costs
Make more money on each trade

I’ve been working on Dominari since around September or October of last year. After racking my brain for months, I more or less wrote off the idea of improving the trade profitability.

That suddenly changed last week on Friday after the market closed. The best reason to trade my own systems live is that the agony of underperforming forces creativity. The feeling reminds me a lot of Daymond John’s (the guy from Shark Tank) new book the Power of Broke. When life isn’t going your way, it’s the resourceful and creative who are best able to get to the top.

Nobody wants to feel broke or under extreme stress. As much as we hate those feelings, they’re often the strongest drivers of performance. That’s how I feel right now with Dominari. I’m so close to getting there and wasn’t sure how to fix that missing ingredient.

If it weren’t for that stress, I would not have had my simple but very powerful insight last Friday.

And please don’t laugh. The change is so dumb and obvious that you’re going to wonder what’s wrong with me. When you’re in the thick of designing a system, the ugly truth is that sometimes you get lost in the weeds. Or to use another botany metaphor, you only see the trees instead of the forest.

My key insight was to slightly modify the exit strategy to use limit orders, whereas previously I only exited based on the close of the bar. I noticed two repeated behaviors that finally beat me over the head enough that the point finally sank in.

The number of occasions where my trade closed in the optimal location seemed to be significantly outweighed by the amount of money left on the table. The key insight for me was realizing where to optimally place that limit order. And for those of you on my newsletter, it happens to be closely related to the Auto Take Profit that I’ve been talking about all week.

Backtest assumptions and results

My operating mantra when doing backtests is to minimize the number of assumptions. Spreads for retail traders have changed dramatically from 2008 to today. I remember working as a broker at FXCM when our typical spread on GBPCHF was something like 8-9 pips. I now routinely pay something like 2 pips. It’s impossible to model what happened in the middle without haphazardly guessing.

I find it far more convincing to analyze the raw signal, both on historical and recent market data, then to interpret whether trading costs are likely to be favorable in today’s markets. “Raw signal” is the ideal signal, one which assumes perfect execution, no slippage, no rollover, no spreads and no commissions. The natural result is that you’re overstating historical performance, but the benefit is that you have a very clear idea whether the core idea is a system capable of predicting the market with reasonable risks.

The total leverage employed in the portfolio is 7:1. If I have a $50,000 trading account and held a position in every currency pair in the portfolio, then the notional value of those trades would equal $350,000 (50k * 7).

Another key point is that I used a fixed position size of $12,500 per trade. The size of the trade never increases or decreases during the backtest, which allows me to isolate the impact of the raw signal without adding the variable of money management.

Here are my trade metrics with version 1 of Dominari. Click the images to view them in full size.

The first version of Dominari had a profit factor of 1.26.

After here’s the change with Dominari version 2.0.

My new version of Dominari increases the profit factor to 1.59 with a significantly lower drawdown.

My best case scenario was to hope that the profit factor would jump another 10 points or thereabouts, maybe stretching the profit factor to 1.35 or thereabouts. It’s incredibly exciting to see the edge over breakeven more than double (going from a $0.26 edge to a $0.59 cent edge).

What I’m most excited about is the skew in the returns. Most mean reversion systems look for an edge but are overwhelmed with the impact of losing trades. That was the case with version 1.

The largest losers outweighed the largest winners in version 1.

This new version of Dominari is the very first mean reversion strategy that I’ve ever developed where the winning tails (ie, the biggest winners) nearly equal the losing tails (the biggest losers). It’s almost always the opposite with mean reversion strategies. Said another way, the risk profile of the extreme outcomes significantly improved with version 2.

The impact of the biggest winners is nearly identical to the biggest losers with version 2.

And the metric that most traders care about the most, drawdown, is wildly improved. Version 1 showed a drawdown of 5.72%. The new version is a fraction of that at 1.77%.

Out of sample backtest for Dominari version 2

The out of sample performance is nearly identical to the in sample performance, despite significantly different market conditions.

When I walked my test out of sample onto recent data, covering 2013-2015, the performance characteristics of version 2 are nearly identical to the in-sample test. The profit factor was identical at 1.59, and the max drawdown was 2.01% for 2013-2015.

Translating the theoretical into expected performance parameters

Again, those metrics above are in the ideal world of perfect execution and no trading costs. The real world performance will have lower returns and higher drawdowns. The advantage to having live trade data is that I can now make some kind of intelligent estimate of my expected trade accuracy and profit factor. Just how overstated are the idealized returns likely to be?

The process that I went through to calculate the expected profit factor in the real world is a 5 step process. I don’t think it’s going to make any sense if I try to write out the steps in conversational English. Instead, I’ve chosen to share a spreadsheet where you can view the step by step process for how extrapolating live trading data into expected performance with the new strategy. Click here to view the spreadsheet.

The expected profit factor for my live trading is expected to be between 1.29 to 1.39. The expected percent accuracy for live trades should jump from 62.55% to 70.8%.

The traders who will get first crack at the Total Access Apprenticeship are those are subscribed to the free newsletter. If you’re not signed up, make sure to fill in your email address in the orange box at the top right of this page.

Engineers Can Leverage Their Experience to Trading

February 25, 2016 by Lior Alkalay 5 Comments

There are many kinds of engineers in the world. In fact, given the vast variety, it would be an effort in futility to try and name them all. But what does it really mean to be an engineer? And the bigger question: how exactly is this related to trading? A good friend of mine who is an engineer said, very aptly, “Scientists do the work of God; engineers do the work of man.”

Although I am not sure he can claim that quote as original, there is nothing better to describe what engineers do. Being an engineer is a mindset. You must be able to solve problems and build mechanisms in the imperfect world of man. And often times, those conditions are far from ideal.

And that is exactly the state of mind you need to build a profitable trading strategy, whether discretionary or algorithmic. So, if you are an engineer, that’s great! This article could help leverage your experience in trading. And even if you’re not, you might still learn something useful.

Engineers define the project in detail

If I were to ask an ordinary trader what trading strategy he plans to build, what might be the answer? More than likely I’d hear that he wants to build a strategy that is profitable. And while that it is, naturally, the ultimate goal it’s hardly enough to build a successful strategy.

If I were to ask an engineer the same question, however, I’d surely get a very different answer. Engineers define each project they’re working on in great detail. In our case, then, the trading strategy they’d likely wish to build would have a number of inputs. An engineer’s definition of a strategy might be something like this:

Strategy Interval: 1-3 Days

Type: Momentum

Maximal Drawdown: 20%

Value at Risk: $500

Target Profit Limit: 100 to 150 pips

Target Stop Loss per Trade: 50 to 75 pips

Target Win Ratio: 50%

By having these very detailed parameters, the engineer already knows several things. First, that he has to design a trading strategy that’s built on momentum (rather than something else). Second, it has to have a maximal draw down of 20% with a maximum risk of $500 at any given time. Third, each trade has to have the potential of at least 100 pips and a stop loss of maximum 75 pips. And fourth, in the longer run, at least 50% of all trades must be profitable.

Clearly, an engineer doesn’t choose a trading strategy arbitrarily. He already knows exactly what he wants to build and chooses all the tools (moving average, etc.) and strategies accordingly. Sure, it does not guarantee he will be successful. And he may eventually have no choice but to change his targets. After all, he is doing the work of man where nothing is perfect.

But the point is that setting a plan in detail narrows irrelevant options. Therefore, everything is more result-oriented.

Engineers anticipate what could go wrong

Most people, when they have a good trading strategy, think about how much money they can make. You know, they dream about how good it can get. But an Engineer’s natural instinct is just the opposite. Instead, an Engineer is trying to figure what could go wrong.

An aeronautical engineer would test an airplane in extreme weather to ensure it doesn’t crash. Similarly, an engineer building a trading strategy will test it for failures. The end result, of course, is that the engineer will build a better and safer strategy.

Engineers are open to ideas

When there is a particular mechanism of functionality to be filled an engineer looks first for a readymade mechanism. In trading, for example, it could be finding a good tool to identify volatility. Or perhaps a good signal for momentum. Once found, provided that it is acceptable, an engineer will incorporate it in his strategy.

An engineer will not be fixated on investing his own idea. Engineers do not believe in re-inventing the wheel. Simply put; if it works, it works.

In Conclusion

Many engineers might now realize they already have the mental tools needed to succeed in trading. They just need to implement their ingrained work habits and apply them properly. And many successful traders will find that they have already implemented the lessons learned here.

Engineer or not, successful trader or not, it’s never too late to learn how to think “smarter.” In the long run, it will enhance your trading strategy.

Monitoring my live trade execution

February 10, 2016 by Shaun Overton 29 Comments

Dominari’s biggest risk is its trading costs. In the midst of losing 6 days in a row, I found myself extremely concerned about Dominari’s performance. Did the signals go bad all of a sudden or is this a normal drawdown? Is Dominari losing because of trading costs?

I decided to start analyzing my FXCM account. Part of the nerves were driven by the fact that it took 2 weeks to setup the account. The compliance process took far longer than usual because I’m a former employee. Two weeks later, I turned on the account just in time to a) miss the biggest equity growth and b) to catch the biggest drawdown.

I felt more hostile to the FXCM account performance because I didn’t have any profits to pad the losses. This is all coming from my original risk capital. And I’m having my third child soon. Giving birth to kids in the US is incredibly expensive. I’ve got better uses for the money than to throw it away in the markets!

So, the real question is: am I losing because it’s just a rough patch or because FXCM is eating my lunch?

backtest-equity-fxcm

This image is a backtested equity curve over the same period of my live performance. I’ve traded live since January 28, but the trading didn’t begin until the afternoon. As you can see, I again missed another patch of strong performance.

The rest shows something of a fairy tale. The backtest shows a return of 19.13% over that period, whereas my live performance is down 10%. How much of that is due to commissions, spread, rollover and slippage?

The backtest shows a profit of $956.65 with no trading costs.

My real results, which 1) show a profit on the backtest but 2) are actually showing a loss in real life, can be used to estimate a floor for my trading costs. The formula for that is
( Total profit and loss + commissions + rollover) / total trades, which is currently $1.58 in costs per trade.

The commissions and rollover are easy to separate out using either Myfxbook or the FXCM account report. The grand total spent so far on commissions is -$239.80 and -$3.05 on rollover.

The hardest part to separate is the spread paid. I’m not recording the spread paid on every trade (maybe that’s a mistake and I need to add it). But I’m going to use the table below to estimate. I took a random sample of 30 trades from the 501 trades completed at the time of my analysis.

Spread Paid	Slippage
0.000198523	1.49E-05
0.000153951	-5.13E-05
0.000455823	0.000227912
9.98E-05	0
0.000161242	-0.00313413
2.76E-05	-9.19E-06
5.55E-05	6.94E-06
0.000110898	-1.01E-05
9.24E-05	0
9.91E-05	-1.57E-16
6.55E-05	1.31E-05
4.85E-05	2.08E-05
8.22E-05	-1.67E-16
6.87E-05	0
6.95E-05	-1.65E-16
0.00015173	-2.17E-05
9.43E-05	-2.36E-05
9.38E-05	-0.00225922
7.61E-05	-0.0024735
0.000160038	1.00E-05
0.00013502	0
0.003542625	4.52E-05
0.000222978	-0.00376275
7.62E-05	0
0.000432797	7.73E-06
2.61E-05	0

The average slippage (the right column) is a stunning -0.044%. I’m getting negative slippage on average with FXCM. That’s outstanding! FXCM is improving my fills even though my entries are requested at a worse price. Whatever misgivings I’ve had about FXCM in the past are alleviated. That’s impressive execution.

Estimating the spread paid is much more difficult. I’ve chosen to take my average trade profit on a $5,000 account as the starting point. The trouble is that the value of an average winner can depend on the account performance. If I use stagnant position sizing, then the drawdown doesn’t effect the value of the average winner. Under that assumption, the average winner is $3.48 per trade.

But if I use compound position sizing, the drawdown eats away most of the profits. That drops the average trade value down to $1.70.

I converted the spread paid from pips into percentages. Using EURUSD as an example, a 1 pip spread works out to 0.0001/1.12727 = 0.000089. The reason for doing this is so that I can compare the spread on EURUSD to something with a much wider spread like AUDNZD. The spread is wider on AUDNZD, but the value of a NZD pip isn’t the same as a USD pip. Percentages allow for an apples to apples comparison.

The average spread paid in my sample was 0.00026157605, which is 0.026%. Putting that back into terms relative to my account balance, I’m paying 0.026% * $5,000 = $1.31 per trade in spread. Across 420 trades, that’s -$550.20 in spreads.

Total costs are spread, commissions and rollover:
$550.20 + $239.80 + $3.05 = $793.05

On a per trade basis, that is $1.78 in costs per trade from my estimates.

The total profit on the backtest was $956.65, but I missed about $550 of it because trading didn’t start until 17:00 on the 28th of January. That leaves the backtest profit somewhere around $406.65.

That puts the re-estimated profit and loss at $406.65-$793.05 = -$386.40. The actual loss is -$469, which I feel is a reasonable discrepancy based on the fact that I’m estimating how much profit was contributed on January 28 instead of knowing for certain.

The conclusion is that I need to turn off this trading at FXCM. Even if I joined their active trader program and traded in the top tier, it would only save me half the commissions. Most of the trading costs are in the spread and not commissions. I’m seriously considering a move to a broker that will allow me to make a market by posting limit orders. But first, I’ll need to go over my Pepperstone account to review the trading costs for myself and clients.

Live demo testing a new strategy with limit orders

November 24, 2015 by Shaun Overton 17 Comments

I come up with amazing looking backtests all the time. This is the latest example using the SB score.

The equity curve of the new strategy without trading costs.

The free and hypothetical version of the strategy yielded $79,618.82 for an uncompounded return of 796.19% over a period of 3 years. The strategy trades all major FX crosses. As you can tell, the signal quality remains nearly constant across multiple market conditions. It looks great.

The problem is trading costs. It’s always trading costs that make life difficult.

Trading costs drop the profits by 98.22%

I always take a heavily pessimistic view when it comes to assuming trading costs and slippage. It requires a lot of intellectual honesty, but making an effort to avoid rosy assumptions saves a lot of pain and disappointment down the road. The assumptions are really severe on cross currencies where we assume spreads and slippage north of 5 pips.

Performance with pessimistic trading cost assumptions drops to only making $1,000 in profit. The strategy doesn’t need to go in the rubbish bin, but it’s far from ready for prime time. There’s no scenario where it makes sense to trade with market orders.

General characteristics

Average trades per day: 39
Currency pairs traded: 27
Percent accuracy: 66.52%
Style: Mean reversion
Charts: Hourly

How to trade on the cheap

I’m notoriously frugal. One of my fraternity brothers in college still tells stories about me counting loose change and tracking it in MS Money.

That kind of mentality drives my wife crazy… but it’s a real asset for a trader! Traders make their money on the margins like every other business person.

I spent yesterday afternoon coding this new strategy with a slight twist. Instead of paying the spread on every single trade, what if I use limit orders to try and earn the spread?

The current raw spread on EURUSD is 0.3 pips, which is worth $0.03 per microlot. The trading commissions are $0.03 per microlot. If I earn an extra $0.03 per microlot, that at least covers the trading costs. On pairs like NZDCHF where the raw spread is 1 pip, that adds an extra $0.04 ($0.10 – $0.03) per side. I.e., the entry signal makes an extra $0.04 and the exit also makes an extra $0.04 on every single trade.

Even quiet pairs on NZDCHF still exhibit a degree of noise on every bar. I haven’t done any research to back it up, but my subjective experience says that the wicks of 90% or more of bars will be at least as long as the spread is wide.

Traders make their money on the margins like every other business person.

Said another way, if the spread on EURUSD is 0.3 pips, then the difference between the open and low price on 90% of bars should be at least 0.3 pips, too. That’s my assumption, anyway.

An example of twisting the strategy to use limit orders

Say that my signal to enter the market just popped up. The current price for EURUSD is 1.06457 x 1.06462, which is 0.5 pips. The backtests assume that I’ll hit the 1.06462 asking price and pay the spread.

The idea for my test is to set my limit order at 1.06457. Since I’m a retail trader, that means I’m asking the market to move down half a pip before I’ll get to have a position. Requiring a small move in my favor theoretically earns more than jumping into the market with both feet.

Live demo testing begins

I could theoretically model the idea in a backtest, but there are critical assumptions that make it pointless.

1) The average spreads available in my 2009-2011 backtest period were far wider than they are today
2) The spread varies significantly throughout the day. EURUSD is routinely as low as 0.2 pips in the European sesssion, but can easily hit over 1.0 pips in the dullest portions of Asian trading.

The second item could be completely detrimental in a backtest. It’s better to test the idea on a live demo and get something closer to real trading data.

The first 15 hours of live demo testing.

I’m only 15 hours into the test, but at least everything is off to a good start.

The goal for the test is simple: place at least 300 trades in the account. That should only take about 2 weeks since the strategy is so hyperactive.

The criterion for success is equally simple: does the real-time demo trading performance meet or exceed the backtesting performance over the same time period?

I started trading in the evening of November 23, which means that I should hit my 300 trade threshold around the 10th day of trading. The trading frequency does fluctuate, but that should occur sometime around December 4th.

Even though I have live demo data, I’m going to run a market entry backtest from November 23 to December 4. If the demo trading, which uses limit orders, exceeds the market entry backtest, then I have a reasonable basis for assuming that the strategy is ready to trade on a small live account.

I’m also ironing out bugs that appear during the live simulation. More than likely, these dates will be pushed back. I already found 2 issues that require investigation after only 22 trades. There’s no point in judging a strategy if it’s not performing exactly as specified.

Code the same strategy twice?

You probably noticed that the forward test equity curve is from MetaTrader. Why would I test in one platform but execute in another? All of my backtests were done in Seer.

If you have two people work on a problem and they both arrive at the same answer, then they probably answered the problem correctly. The same logic applies to programming. If I program a version of the strategy and Jingwei programs a version of the strategy, they’re supposed to place the exact same trades. Any discrepancies mean that someone’s programming is wrong.

I routinely use this method because the slightest errors in logic can lead to dramatically different trading outcomes. It’s the difference between making a lot of money and losing a lot of money. Yes, I’m sacrificing efficiency. The stakes for a strategy are so high that it’s better to make 2 people do the same work in exchange for the confidence of knowing that it was done properly.

MetaTrader is inferior to Seer by every measure. The only reason that I wrote my code in MetaTrader was that I’m anxious to test the idea. MQL4 is easy for me to code – programming for MetaTrader is one of our main services.

After Jingwei finishes programming the Seer version next week (she’s off for Thanksgiving), I’ll have the basis for comparing my MT4 version against hers. It’s terribly inefficient, but I also know how likely I am to waste weeks on analyzing trades placed according to rules that don’t exactly match my strategy. Better safe than sorry!!!

How to fatten the margins

One thing I hate about retail trading is that very few venues offer a true ECN. Trading on a traditional retail forex broker means that I have to wait for the spread come down to touch my order. In the example I gave using EURUSD, it requires that the market move 0.5 pips in my favor before I get a fill.

Trading on an ECN would significantly increase the probability of receiving a fill on the limit order. Using the EURUSD example where the current prices are 1.06457 x 1.06462, I would place a buy limit order on the bid at 1.06457. If anyone in the market sells at that time, it means that at least a portion of the order would be filled almost immediately.

In effect, trading on the retail spreads contains the worst case scenario for execution. The price has to adjust 0.5 pips in your favor in order to get filled. If you trade on an ECN and the price fell 0.5 pips, you would get filled every single time. But you also get the chance to get filled earlier and faster because if anyone comes in and goes short at market, the order sits on the book waiting for someone to hit it.

Smart traders do everything in their power to fatten up the margins

I’m proceeding with the demo test now. If it meets or exceeds the backtest results, I’ll then know with the highest degree of confidence possible that the method is ready for live trading. I’ll probably start with a few thousand dollars for the first month. Then, if it succeeds, I’ll really start to scale it.

There’s no reason that all trades must occur on H1 charts. I can always shift the trading intervals by one minute, two minutes… fifty-nine minutes. And even there, it’s possible

My ideal scenario is to trade the strategy on an ECN venue, which requires a minimum balance of $250,000. That amount of money is far higher than I’m comfortable risking. The old rule of trading is that you never risk more than you’re comfortable losing.

That means I’ll likely be looking for a partner to make sure the strategy runs in the best environment possible (an ECN). Are you possibly that partner? If so, send an email to info@onestepremoved.com and introduce yourself. Nothing will happen for several months, but it always takes awhile to build relationships and feel comfortable with a project.