Algorithmic and Mechanical Forex Strategies | OneStepRemoved

One line of code makes all the difference

by 4 Comments

I was really excited about my Pilum strategy two months ago. The research looked great and everything was ready to rock and roll. Demo testing began and then… not much happened.

The Quantilator is (mostly) finished, which finally gave me time to circle back and review what happened with Pilum.

Live demo trading of Pilum

Live demo trading of Pilum. Dec 9, 2016 to Feb 7, 2017

The expected outcome was that I would win 75% of the time. Trades were infrequent, so I thought maybe I’m just having bad luck. But then my win rate remained stuck around 50%. Simple statistical tests told me this was unlikely to be bad luck.

I used the research time to pour over my research code and to compare it with live trades. What I found was that a single line of code (AHHHHHHHHHHHHHHH!) was incorrectly calculating my entry price, dramatically overstating the profits.

The flawed code produced this equity curve from a single combination of settings:
Flawed Pilum backtest

When the actual, correct result looks like this with those same settings:

The accurate backtest of Pilum

The accurate backtest of Pilum

I’ll be honest… I like the flawed backtest a lot more!

The new, single-setting backtest isn’t as good, but it’s still trade-worthy. There are some characteristics that I dislike and features that I love. Let’s dig into those.

What I dislike

The frequency of trades is very low. Out of 19 months there were a total of 43 trades. 43 trades to comprise a backtest on 40+ instruments is a very small number.

If it weren’t for the statistical pattern backing up the frequency, I would not consider the test. However, there are 20,000 bars each on the 44 instruments. There are 880,000 total bars used to analyze whether my Pilum pattern offers any predictive value.

The most valuable predictions, however, are also exceptionally rare. That’s why I’m not able to get the trading frequency higher, which would potentially smooth the returns.

What I love

My previous systems like QB Pro and Dominari traded actively for relatively small wins. Trading costs exercised a massive impact on the overall performance.

The accurate backtest of Pilum

The accurate backtest of Pilum

Now look again at the correct equity curve (the image to the right). Do you see the final profit of roughly 0.14? That’s a 14% unleveraged return over a 19 month period.

Allocating 2:1 or 3:1 leverage on this strategy could average annual returns of 15-25%.

Detecting hidden risk

A key measure of risk is skewness. You may not use that term yourself, but it’s something most of you already understand. The biggest complaint about people trading Dominari was that the average winner relative to the average loser was heavily skewed towards the losers.

Dominari wins on most months, but when it lost in December it was devastating. I implemented what I thought was a portfolio stop after the December 9th aftermath. Then I had a smaller, but still very painful, loss in January. The portfolio level stop loss of 3% should prevent future blowouts now that I know what goes wrong.

I still believe in Dominari. But, I obviously lost the work of most of the year due to those events.

Knowing that skewness is a good measure of blowout risk (even if you’ve never seen it in a backtest, like happened with Dominari), Pilum looks extremely encouraging.

This is a histogram of profit and loss by days. You should notice a few things.

The tallest bar is to the right of 0. That means that the most frequent outcome is winning.

worst and best days

The biggest winning day is dramatically better than the worst losing day. The worst outcome was a loss of 2%. The best outcome is gains near 10% in a single day (unleveraged!).

This is the statistical profile of an idea that’s much more likely to grab an avalanche of profits than it is to get blown out.

It gets even better

low correlation

Would you say that the blue and red equity curves are highly or loosely correlated? Look closely.

Writing this blog post made me think carefully about the Pilum strategy. I decided that maybe I should see if all of the profits are coming from different settings at the same time. There’s very little risk of overfitting the data as my strategy only has 1 degree of freedom.

The blue bars are the equity curve of Setting 1.

The red bars are for Setting 2.

Do you think these are tightly or loosely correlated?

If you said loosely correlated, then you are correct. Notice how each equity curve shows large jumps of profit. Did you notice how those profit jumps occur on different days?

The blue setting skyrockets on a single day in November 2016. It leaves the red equity curve choking in its dust.

But then, look what happens as I advance into December. The red curve dramatically catches up to the blue curve and even overtakes it.

The correlation between the 2 strategies is only 57%.

Combine multiple settings into 1 portfolio

Combined settings Pilum equity curve

This is a much nicer equity curve!

Loose correlations are a GIFT. Combining two bumpy equity curves into a single strategy makes the performance much, much smoother.

The percentages of days that are profitable also increases. Setting 1 is profitable on 58.0% of days. Setting 2 is profitable on 53.5% of days.

But… combining them makes Pilum profitable on 68.2% of days. Awesome!

That also provides more data, which puts me in a stronger position to analyze the strategy’s skewness. Look at the frequency histograms below. They’re the same type of histograms that I showed you in the first section of this blog post. As you’ll notice, they look a lot different.

Pilum most probable daily profit and loss

The most probable outcome for any given day is a small winner

The tall green bar is the most probable trading outcome for any given day with filled orders. The average day is a positive return of 0-1%.

The small red bar is the worst trading day of the combined strategy.

The small green bars are the best trading days of the combined strategy.

Look how far to the right the green bars go. The largest winner is more than 3x the biggest loss. And, there are so many more large winners compared to losers.

Giant winners are far more likely than comparable losses.

The Plan

I immediately pushed Pilum into live trading this combination of two strategies. I expect that adding a second degree of freedom and running about 30 different versions of the strategy – all with different settings – will add to the performance and smooth the returns even further.

Dominari hasn’t been working on my FXCM account, which is very difficult to accept because the lacking performance seems to be a buried execution issue. Pilum, however, trades very infrequently. It’s unlikely that execution quality will make a dramatic difference in the long term outcomes.

So, I’m going to convert the FXCM account to trading Pilum exclusively. That will be offered as a strategy on Collective2 within the next few weeks, a company with whom I’ve been working closely. Their users are more investor rather than trading oriented – they’re far more likely to view low trading frequency as a good thing. I suspect that most people here have a different opinion and want to see a lot of market action.

I’ll write an update on Dominari shortly.

Flat and happy

by Leave a Comment

This is the first financial event since 2008 that’s hit the mainstream public. Even my friends from college are talking about the Brexit on Facebook.

My Dominari system only trades during the UK evening, so I felt comfortable leaving my system on overnight. When I woke up, however, I didn’t feel the same. Did you see the GBPUSD chart? Holy cow! 1,300 pips in an hour.

Brexit

GBPUSD lost more than 1,790 pips in a day from top to bottom on the Brexit.

This is the first time I’ve intervened in a trading system since April of last year. What makes me very happy, though, is that this intervention is all about protecting profits. I’m up 6.69% since I began trading the finalized version of Dominari on April 15.

Dominari equity curve

My equity curve as of June 24, 2016.

myfxbook.com/members/QuantBar/dominari-pepperstone/1591822 – my results at Pepperstone

Dominari isn’t intended to trade these types of markets. So, instead of deciding to “see what happens”, I’m flat and happy until we see how the markets open after the weekend. I expect big gaps. I don’t feel like gambling which way the gaps may go.

If you clicked the original link, you noticed that the equity curve is marching straight up. That’s what’s supposed to happen. But like any good system trader, I wanted to see it working in the real world before I upped the capital commitment.

Earlier this month, I decided to trade a second account at FXCM, this time in USD. That brings my total accounts to €8,500 and $5,100. That’s about $14,600 in USD terms between the two accounts.

The FXCM account started live trading on June 6. Before then, I made sure to test it on an FXCM demo account to confirm that my edge wasn’t completely dependent upon broker selection. I’m happy to report that the FXCM results are closely mirroring those at Pepperstone.

myfxbook.com/members/QuantBar/dominari-fxcm-mt4/1679763 – my results at FXCM.

43 million real trades reveal the tactic of profitable forex traders

by 4 Comments

Traders that follow one simple rule are 3.118 times more likely to be profitable 12 months later than those that don’t.

The critical feature of profitable traders is their reward to risk ratio. Yes, you’ve probably read that before, but this time it’s backed up with research. FXCM studied 43 million real trades from traders around the world to produce this analysis.

Image credit: DailyFX

Image credit: DailyFX

Everyone “knows” that 90-95% of traders lose money. The good news is that the real percentage is noticeably lower. 83% of all traders lose money. And, that’s among the worst group. When traders use a reward to risk ratio of 1 or more, 50% of all traders are profitable after 12 months.

Be warned: the phrase “correlation is not causation” very much applies here. I cannot promise you that based on the data that using reward to risk ratios greater than 1 will automatically give you 50-50 odds of being profitable in the long run.

Logic, however, suggests that using good reward to risk ratios is a good idea. The advice to use reward-risk ratios above one appears in every trading book ever written for a good reason.

When traders use a reward to risk ratio of 1 or more, 50% of all traders are profitable after 12 months.

I suspect that it’s not the ratio itself that’s important. Instead, a large ratio discourages the worst mistakes that traders make.

I remember a project when I worked as a broker at FXCM. The systems desk analyzed the trades of the company’s most consistent losing traders. Perhaps taking the opposite signal of the worst traders might lead to profitable trades?

Alas, we found something far more mundane: the worst traders lose because they over-trade.

Trading costs

Think about how trading costs apply to the reward risk ratio. If you earn $2 for every $1 that you lose, it makes scalping an impossible activity

Traders using a 2:1 ratio need to use more patience. Even though FXCM offers low spreads and commissions, a 2:1 reward risk ratio implies further distances to the profit target. Longer pip distances lower the cost of every pip of profit.

Cost examples

FXCM averages a 1.4 pip spread on EURUSD. Let’s see how our reward-risk ratio affects trading costs using the 1.4 pip spread for our 2 examples.

Scalping

Profit target: 10 pips
Spread: 1.4 pips
Spread as a percentage of the profit target: 14%

Intraday Trend Trading

Profit target: 50 pips
Spread: 1.4 pips
Spread as a percentage of the profit target: 2.8%

Your cost as a percentage of profit in these examples are 5x higher when you scalp. That’s not good!

Holding trades with bigger profit targets minimizes the impact of trading costs. Said another way, you get to keep more pips when you win by increasing the distance of your profit target from your entry price.

The advice to use reward-risk ratios above one appears in every trading book ever written for a good reason.

Following a reward risk ratio greater than 1 naturally pushes you towards lower trading costs. Lowering your trading costs logically suggests you have a higher likelihood of long term profitability. If you want to get other critical tips for similar results, then make sure to sign up for the Foundations of Profitable Trading Checklist.

Reward risk ratio explained

The reward risk ratio compares your average profit to your average loss. If your average winning trade is $30 and your average losing trading is $15, then you have a reward risk ratio of 2:1. If your average winning trade is only $8, but your average losing trade is $16, then your reward risk ratio is 0.5:1.

Does the winning percentage matter?

Amazingly, the percentage of winning trades doesn’t seem to matter. The high frequency trading firm Virtu is a great example of this. Virtu wins on 99.999% of trading days even though it only wins on 49% of its trades.

The FXCM data shows that the average trader wins more than 50% of the time. EURUSD trades won 61% of the time, while some pairs were closer to 50%. The percentage of winning trades on all currency pairs is greater than 50%.

win loss percentage by forex pair

Image credit: DailyFX

Despite winning more than 50% of the time, trades with a poor reward risk ratio only had a 17% chance of earning a profit 12 months later.

… you get to keep more pips when you win by increasing the distance of your profit target from your entry price

If you’re currently struggling with your profitability, you’ve probably thought to yourself, “I need to win on more of my trades.” It’s like a business owner saying, “I need more customers.”

Smart business owners know that finding more customers is time consuming and expensive. It’s often much easier to sell more stuff to the customers that you already have.

It works the same way in trading. Instead of worrying about winning more often, you should focus your efforts on squeezing a few extra pips out of your winning trades.

If there’s anything that you should learn from this research, it’s this: the fastest way to improve is to earn more pips on your winning trades. You do not need more winning trades to do better.

Types of strategies with good reward risk ratios

The type of strategy that you select almost automatically dictates your reward risk ratio. Ranging strategies usually have ratios less than 1, which the FXCM data shows have a 17% likelihood of long term profitability. Trending strategies have ratios greater than 1, which have 50% probabilities of long term profitability.

Ranging strategies

If you daytrade EURUSD where the daily range has recently been around 80 pips, then that 80 pip range is the hard ceiling of what you could possibly make in a day. You know from experience that getting the bottom tick or the top tick of the day almost never happens. If you’re lucky, you may enter within 10-20 ticks from the bottom.

Upon entry, you also need to give the trade breathing room. That stop loss probably needs to be something like 25 pips if it’s a tight stop or 40 pips in order to have plenty of breathing room.

The best exits in a ranging market occur in the middle. You don’t know if the market will push back to its ceiling. It has just as much chance as going back to support and it does up to resistance.

The mid point of an 80 pip range is 40 pips, but you’re likely entering 10-20 pips from the true bottom. That only gives you a potential range of profit targets from 20-30 pips.

The most realistic, good ratio is a 30 pip profit target on a 25 pip stop loss, which is 1.2. Most strategies will probably risk 40 pips to make 20, which is a ratio of only 0.67.

Consider what a range trading strategy is. The market is stuck. It’s having a hard time going anywhere. You should only range trade if you have a well researched strategy with a long term edge. Otherwise, the typical trader is 83% likely to walk away with losses after a year.

Trending strategies

Trend trading strategies should last for weeks or months at a time. Looking again at EURUSD on a multi-month time frame, the current long term range is from 1.05 up to 1.16. That’s a range of 900 pips, but it’s not like the market wobbles up and down through that range. Instead, it gets stuck near 108, then briefly pushes down. It comes back to 1.08, then pushes up to 1.12. It might push up again to 1.15, then trade back down to 1.08. It’s hard to guess whether the next move will be up or down.

long term trend

A 3,498.4 pip move in the EURUSD over a 10 month period.

Better long term plays are to sit on trades and let them pick a direction. The best recent EURUSD example began on May 8, 2014 at 1.39934 and ended March 13, 2015, at 1.04946. That’s a colossal 3,498.4 pip move in just 10 months.

Is there a scenario where you’ll risk almost 4,000 pips on a trade? Of course not. What about 1,000? No! What about 500? No!

The natural risk reward ratio for these types of trends is astronomically high. For a few hundred pips of risk, you can make 10 or more pips for every one risked.

As long as you’re not aggressively trading, trending strategies are far more difficult to mess up. If you can click a button, enter a stop loss and then do nothing for months at a time, then you’re qualified to consider trend trading.

The practical application is of course more difficult than that description, but that’s the idea in a nutshell. If you’re a newbie forex trader and wondering where to start, long term trends are the place where you’re less likely to get hurt.

The problem for newbies, though, is that they’re looking for excitement. It’s not terribly exciting to place on trade and then do nothing for months. It’s one of the paradoxes of the market that less work can often lead to better results.

How to improve your trading

The reward to risk ratio is a critical element for new traders to increase their chances of success, but it’s not the only one. Click here to register for our free Foundations of Profitable Trading Checklist. You’ll learn simple, but useful, tips to improve your trading.

Monitoring my live trade execution

by 29 Comments

Dominari’s biggest risk is its trading costs. In the midst of losing 6 days in a row, I found myself extremely concerned about Dominari’s performance. Did the signals go bad all of a sudden or is this a normal drawdown? Is Dominari losing because of trading costs?

I decided to start analyzing my FXCM account. Part of the nerves were driven by the fact that it took 2 weeks to setup the account. The compliance process took far longer than usual because I’m a former employee. Two weeks later, I turned on the account just in time to a) miss the biggest equity growth and b) to catch the biggest drawdown.

I felt more hostile to the FXCM account performance because I didn’t have any profits to pad the losses. This is all coming from my original risk capital. And I’m having my third child soon. Giving birth to kids in the US is incredibly expensive. I’ve got better uses for the money than to throw it away in the markets!

So, the real question is: am I losing because it’s just a rough patch or because FXCM is eating my lunch?

backtest-equity-fxcm

This image is a backtested equity curve over the same period of my live performance. I’ve traded live since January 28, but the trading didn’t begin until the afternoon. As you can see, I again missed another patch of strong performance.

The rest shows something of a fairy tale. The backtest shows a return of 19.13% over that period, whereas my live performance is down 10%. How much of that is due to commissions, spread, rollover and slippage?

The backtest shows a profit of $956.65 with no trading costs.

My real results, which 1) show a profit on the backtest but 2) are actually showing a loss in real life, can be used to estimate a floor for my trading costs. The formula for that is
( Total profit and loss + commissions + rollover) / total trades, which is currently $1.58 in costs per trade.

The commissions and rollover are easy to separate out using either Myfxbook or the FXCM account report. The grand total spent so far on commissions is -$239.80 and -$3.05 on rollover.

The hardest part to separate is the spread paid. I’m not recording the spread paid on every trade (maybe that’s a mistake and I need to add it). But I’m going to use the table below to estimate. I took a random sample of 30 trades from the 501 trades completed at the time of my analysis.

Spread PaidSlippage
0.0001985231.49E-05
0.000153951-5.13E-05
0.0004558230.000227912
9.98E-050
0.000161242-0.00313413
2.76E-05-9.19E-06
5.55E-056.94E-06
0.000110898-1.01E-05
9.24E-050
9.91E-05-1.57E-16
6.55E-051.31E-05
4.85E-052.08E-05
8.22E-05-1.67E-16
6.87E-050
6.95E-05-1.65E-16
0.00015173-2.17E-05
9.43E-05-2.36E-05
9.38E-05-0.00225922
7.61E-05-0.0024735
0.0001600381.00E-05
0.000135020
0.0035426254.52E-05
0.000222978-0.00376275
7.62E-050
0.0004327977.73E-06
2.61E-050

The average slippage (the right column) is a stunning -0.044%. I’m getting negative slippage on average with FXCM. That’s outstanding! FXCM is improving my fills even though my entries are requested at a worse price. Whatever misgivings I’ve had about FXCM in the past are alleviated. That’s impressive execution.

Estimating the spread paid is much more difficult. I’ve chosen to take my average trade profit on a $5,000 account as the starting point. The trouble is that the value of an average winner can depend on the account performance. If I use stagnant position sizing, then the drawdown doesn’t effect the value of the average winner. Under that assumption, the average winner is $3.48 per trade.

But if I use compound position sizing, the drawdown eats away most of the profits. That drops the average trade value down to $1.70.

I converted the spread paid from pips into percentages. Using EURUSD as an example, a 1 pip spread works out to 0.0001/1.12727 = 0.000089. The reason for doing this is so that I can compare the spread on EURUSD to something with a much wider spread like AUDNZD. The spread is wider on AUDNZD, but the value of a NZD pip isn’t the same as a USD pip. Percentages allow for an apples to apples comparison.

The average spread paid in my sample was 0.00026157605, which is 0.026%. Putting that back into terms relative to my account balance, I’m paying 0.026% * $5,000 = $1.31 per trade in spread. Across 420 trades, that’s -$550.20 in spreads.

Total costs are spread, commissions and rollover:
$550.20 + $239.80 + $3.05 = $793.05

On a per trade basis, that is $1.78 in costs per trade from my estimates.

The total profit on the backtest was $956.65, but I missed about $550 of it because trading didn’t start until 17:00 on the 28th of January. That leaves the backtest profit somewhere around $406.65.

That puts the re-estimated profit and loss at $406.65-$793.05 = -$386.40. The actual loss is -$469, which I feel is a reasonable discrepancy based on the fact that I’m estimating how much profit was contributed on January 28 instead of knowing for certain.

The conclusion is that I need to turn off this trading at FXCM. Even if I joined their active trader program and traded in the top tier, it would only save me half the commissions. Most of the trading costs are in the spread and not commissions. I’m seriously considering a move to a broker that will allow me to make a market by posting limit orders. But first, I’ll need to go over my Pepperstone account to review the trading costs for myself and clients.

Resetting my demo testing

by 6 Comments

I mentioned in my new strategy post that live demo testing would last for two weeks unless bugs popped up. Bugs did pop up… but only in my rush to get ready for Thanksgiving!

I sat in my office frantically trying to wrap up code before business hours ran out on the day before Thanksgiving. The plan was to unplug for the long weekend. And, I did. Emails went on autoresponder and, aside from one night binging on Netflix, I didn’t see a screen for 4 days.

Rushing while problem solving, and especially programming, almost never ends well. I would never push untested code changes onto on a live account. But since I figured the change was simple and it’s only a demo account, it’s not a big deal if I mess it up. There went 4 days of live market testing down the drain.

The bug

I’m testing my live demo on FXCM US. As most US traders are aware, the FIFO rules imposed on FX brokers can get pretty annoying. I noticed that a small handful of my orders were being rejected in a special circumstance.

Say that I’m in a long trade and place my take profit at 1.0550. Now say that the signal is so strong that not only should I exit long, but I should also go short at 1.0550.

Anyone in a normal market would place a limit order to sell double the open position at 1.0550. That would cause the long trade to exit and a new trade to open in the opposite direction.

FIFO rules prohibit this because I’m already long. My intended bug fix was to place a take profit order to exit. The order to go short would then remain hidden in the code until the long trade exits. Doing the order logic this way follows my original intent, but also complies with FIFO.

The buggy bug fix

Using my long and then going short example, my long sets a take profit and then the code was intended to send 1 limit order to go short. I forgot to tell the code that it had already sent that one order. Instead, the code sent duplicate orders on every single tick.

Position sizes of 1 microlot mushroomed into several standard lots. Needless to say, the profit and loss began to swing wildly. The massive jump in size also swamped the statistics that I was hoping to measure with the data gathered so far.

It’s easier to start a new demo account, which is what I’ve done as of 5 minutes ago. One upside to using a new demo account is that I can run the code on an account size that matches my intended live testing balance. I plan to test with $2,000 instead of the $50,000 in the first demo.

The first few days of trading

equity curve

This was the first 3 days of trading

The first few days continued much in the same vein as in my original post. I’m looking forward to collecting data from the new demo account. You can expect to stay up to date on my progress if you’re a subscriber to my free newsletter.

How I picked my brokers

Using limit orders is extremely price sensitive. If you place a limit order to buy EURUSD at 1.0550, the price must exactly hit 1.0550 in order for that trade to execute.

One danger of trading on marked up spreads is that you don’t know whether the broker is widening the bid or the ask. If the interbank market quotes a 0.3 pip spread on EURUSD at 1.05480 x 1.05483 and the broker usually charges a 1.3 pip spread, then the broker can mark up the spread in several different ways.

  • Add the 1 pip markup entirely to the ask. The price on your screen appears as 1.05480 x 1.05493
  • Add the 1 pip markup entirely to the bid. The price on your screen appears as 1.05470 x 1.05483
  • Split the 1 pip markup across the bid and the ask. The price on your screen might be as 1.05475 x 1.05488

I of course have no idea which way the broker marks up the spread. If they’re smart, they will price shade in order to earn extra income from the easy order flow. There’s always the risk that the markup could cause one of my orders to not get filled, even though the real interbank market actually touched that price.

I’m doing my demo test at FXCM for three reasons.

  1. Commission only trading. It’s a pure interbank feed, so I don’t have to worry about how spread markups affect my execution.
  2. The commissions are very fair for a retail trader. It works out to $60 per side per million, which is only double the commission that a small institutional trader would pay. Retail rading costs have fallen dramatically in the past few years.
  3. Seer is already plugged into FXCM

When it comes to trade live, I’m planning to trade a personal account at FXCM US and a second account denominated in euros for my Irish company, Dominari (the same one that owns QB Pro). Dominari’s trading will go through Pepperstone, which offers the same commission only setup but 1) charges even lower commissions in euros and 2) offers much higher leverage.

Did your broker go bankrupt this morning?

by 6 Comments

It’s an absolute bloodbath in the FX markets. The tide’s gone out and it’s now very apparent who has good risk management systems in place and who was reckless safeguarding your deposits.

Is your broker on this list?

FX brokers get slaughtered

FX brokers are led to the slaughterhouse.

One of first articles that I send traders on the free EAs list is how to protect yourself from a forex broker bankruptcy. It’s absolutely, critically important where you decide to trade.

I trade at Peppertsone and I strongly recommend that you trade at Pepperstone, too. They made it through this crisis unscathed. They’re well regulated. They’re in a safe and stable banking jurisdiction. And… they’re still running a thriving business.

PS: QB Pro made it through the CHF chaos unscathed. We closed out with a nice profit yesterday.

Automated Trading

by 2 Comments

Nathan Orange contacted me in early 2012 looking for advice about automating a grey box strategy. Through the course of our conversation, it turned out that he was a profitable trader with a multiyear track record. Nathan has gone on to found his own forex signal service at Global Trend Capital.

Nathan conducted this interview with the intention of informing his readers about automated trading. You’ll have to pardon the vanity of publishing his interview of me, but I believe it’s useful for my own readers.

Nathan Orange

 

(Nathan):
Shaun, good to talk to you again and I appreciate you taking the time to discuss what I consider a very important topic. Before we jump into the specific questions, let’s fill everyone in on your background.

 

Shaun Overton(Shaun):
I led the sales effort for the Sentiment Fund at FXCM, which was a fully automated strategy based on the market positioning of retail clients. I needed to understand how it worked in order to answer client questions. That interaction with the systems desk gave me access to one of the tiny handful of people in the forex industry that really knew anything about systems trading and analysis.

I tried trading manually during work hours, but as a broker, it was really difficult to manage trading accounts and to squeeze in 100+ attempted phone calls per day. I also suffered from the usual sob story that every trader endures. Account #1 blew up in 3 months. Account #2 blew up in 6 months. That was the first $5,000 thrown down the pit.

Technical analysis with its trend lines and other tools are hocus pocus pseudo-science. I traded like that for nearly a year, but I never felt confident or comfortable with the idea that subjectively drawing lines on the chart leads to any useful information.

The idea of quantitatively defining a strategy allows for testing and analyzing an idea to determine whether or not it really held any merit. The first non-technical analysis idea I had was to look at unusually big bars with the idea of fading those moves. Access with the FXCM Systems desk helped shape my idea from a subjective idea like “big bar” into a mathematical parameter like “standard deviation”. They also explained trading platforms to consider and recommended a few programmers to help develop the idea.

My experience working with programmers was uniformly terrible. I tend to dive into projects, so rather than depending on the clown-car brigade to half-develop my ideas, I wanted ultimate control over the development process. That eventually led to 20+ hours per week programming and analyzing strategies at home after working all day. The system design bug bit hard and never let go.

Nathan Orange(Nathan):
One of the most common concerns when discussing back-testing is over-optimization. From your perspective, what are some of the common mistakes that most system developers make? I have my own list, but we can discuss those further when we turn the tables.

Shaun Overton(Shaun):
The basic kernel of the idea either has merit or it does not. There is no secret set of magical inputs that turns a bad strategy into a good one. Bad inputs, however, can turn a good strategy into a bad one.

Optimization fails to differentiate between “profitable” and “good”. I flog this dead horse constantly, but the most confusing thing about trading is that you can trade by flipping a coin and setting a 50 pip stop, 50 pip take profit and actually come out a winner – sometimes. Most of the winners will show small profits. A tiny handful of them would show gigantic profits purely as the result of luck. What’s worse is that most of the profitable traders will actually believe that they are the reason for their success when it’s really just dumb luck.

Optimization is usually the process of finding the luckiest accidental winner. It’s no wonder that optimized strategies almost universally fail going forward. The real task is to distinguish between ideas that are inherently non-random versus strategies or expert advisors that coincidentally make money from a random process.

Nathan Orange(Nathan)
Based on your experience and knowledge, if someone sends you a system to code can you quickly determine potential issues with their logic, or even over-optimization red flags? For example, you might a get a system a trader or hedge fund wants coded that has so many specific variables that you know immediately it won’t be robust. I can usually spot these issues from my own system development experience, but from your perspective as a coder is it fairly easy to recognize?

What do you do in those cases? Are most clients bull headed, avoiding any feedback or are they more open minded to listen?

Shaun Overton(Shaun):
We see our primary role as that of a construction worker. If you want to build an ugly house, that’s your affair. On the flip side, if you solicit my opinion, I won’t hold back telling you it’s the ugliest house I’ve ever seen.

People frequently ask, “Do you think this will work?” I almost always answer no, and then they hire us to build it anyway.

Interestingly, strategy development is very similar to trading in that people get emotionally attached to ideas. Even in the face of strong warnings, they charge ahead. A dear friend of mine opined on the subject, saying, “A handful of people don’t try. An even smaller handful listen to good advice. The rest of us learn the hard way.” Most people require the experience of falling flat on their face before they learn the lesson behind the advice.

If you’re motivated enough to ask a programmer to build a strategy for you, it’s because you already know that it is something that you really want to try. I could bluntly say, “This is going to wind up in tears.” 95% of people go ahead with the project, anyway.

Despite my knowledge of markets and systems, I’m not an oracle, either. I’ve told people that I thought their ideas were bad, only to have them come back a year later and tell me they’re making money.

…….Stay tuned for Part II when we discuss HFT, more back-testing issues (including those unique to Metatrader) and if there are common themes to successful systems.

Forex Magnates

by Leave a Comment

I spent all of Wednesday at the Forex Magnates conference in London. Although it’s been around for several years, Forex Magnates is the go-to site for everyone in the forex industry. I ran into a surprising number of colleagues, which came as quite a pleasant surprise.

The presentations on the 10 most unique ideas in the forex industry really caught my attention. One of my favorites was tradable. They must have some massive venture funding. Their site looked spectacularly well done and they employ 37 people despite launching this year. The company has built an open source API usable by both brokers and traders with a social network stacked on top of it.

One of the speakers on an earlier panel mentioned how nearly every industry has some type of open source code that unites how business is done. Banking, for example, uses the SIWFT system for international settlements. The institutional forex side has the FIX protocol for electronic trading. Retail forex, despite roughly 15 years of existence, is still a highly fragmented industry. MetaTrader is the only platform uniting disparate brokers. The sentiment across the room was that everyone except Alpari was grudgingly pushed into using MT4. The other brokers made it clear that they wouldn’t be upset if something else came along. An open source system, which means that you can 100% customize the platform, could be the rival that unseats MetaQuotes. Kudos to tradable for taking the initiative and placing it into a social network.

I also enjoyed the presentation from Rosario Ingargiola, the founder of FxOne. Rosario’s big idea is to help reduce the need for programming by moving into a platform that every major institution uses: Excel. It may surprise most readers to learn that multibillion dollar hedge funds trade using Excel, but it’s true. Excel is such an easy platform that the quants can easily navigate and do their data analysis without any major effort. It’s the lack of programming that attracts them. When they’re ready to launch, they hand the spreadsheets to a programming team to do all the backend wiring for making the trades happen.

FXOne has skipped that step; the backend wiring is already done for you. Rosario’s big idea is to reduce the programming requirements sot hat anyone who can work a basic spreadsheet with formulas can automate a trading system. You might find it unusual for a programming company to promote a system that claims to eliminate programming. In reality, I don’t expect programming would be entirely eliminated. Most people are uncomfortable with logical processes of if-then statements. Most of our customers use us to help organize their ideas. Although most people don’t realize it, I feel that our programming service is most valuable as a “thought organization” process rather than directly as a programming service. I already looked into the FxOne platform about a month ago and have spoken with Rosario on multiple occasions. I expect that we’ll have a role to play with most people who’d like to trade on spreadsheets.

The CEOs from FXCM, Saxo Bank, Alpari and GAIN were all in attendance. As nearly everyone commented, I don’t think we’ve ever seen the heads of our industry all garthered in one spot before. The topic of the CEO panel discussion was mergers and acquisitions, which started gaining attention recently. FXCM launched FXCM Ventures earlier this year with the intention of spotting buyout opportunities. Saxo Bank keeps making headlines through its purchases of white label partners in developing markets.

The CEO group unanimously bemoaned the lack of volatility in the market. Broker profitability directly correlates with market action. Traders don’t trade when the markets are quiet and boring. I had been wondering why the stock of FXCM and GAIN has gotten hammered so badly this past year. Both chalked it up to the decline in trading volume. As Drew Niv noted, his company has increased its number of accounts fivefold in five years, yet profitability is still flat.

FXCM Spreads

by Leave a Comment

FXCM launched a new web site this week. The company shifted its marketing strategy to emphasize lower trading costs. The lower spreads, however, come with a catch. The trader must choose to trade either on a dealing desk or using pass through execution. Leaving the account on pass through execution means paying a whole extra pip on trades.

Dealing desks mean that the company chooses to take risk against its clients. When Joe Trader goes long the EURAUD, the dealing desk simultaneously goes short by selling Joe Trade the EURAUD. Most people forget what a trade is because of the electronic trading environment. It’s easy to associate clicking buttons with “stuff happening” afterwards. Nonetheless, it’s important to remember that you are trading. That is, you are exchanging currencies with someone. A trade is impossible to do without that other someone.

The risk of trading through a dealing desk is that the broker perverts its incentives. The job of a broker is to ensure that his client receives quick and satisfactory execution on all orders. When a company like FXCM chooses to also act as the counterparty to the trade (i.e., a dealing desk), those goals are in direct conflict with the client. Poor execution directly harms Joe Traders while directly benefiting FXCM.

That’s the risk. The vast majority of the time, dealing desks provide quick and appropriate execution to their clients. Egregiously abusive practices would lead to clients closing their accounts and bad mouthing the company all over the internet. The degree to which brokers indulge in questionable execution on trades obviously varies heavily from company to company.

Dealing desks hate scalpers and scalping expert advisors for two reasons. Scalping involves opening and closing trades within a short period of time, usually within a few minutes. This causes headaches for dealing desks because they need to manage the net risk of all open orders. Joe Trader opening and closing microlot trades causes very little harm to the dealing desk. When there are 500 other Joe Traders doing the same thing, however, it becomes a hassle. The net exposure of the order book fluctuates from minute to minute, usually to the dealing desk’s short term disadvantage.

Scalperes work to the advantage of the dealing desk over the long run. Scalpers pay an absolute fortune in spread costs, which quickly and silently drains the account of its equity. Scalpers tend to go through lengthy periods of picking up pennies consistently every day. Then the steamroller comes along and flattens them and the little gains that they accumulated. The dealing desk bleeds while those pretty equity curves are drawn. Then, when volatility picks up, the desks make out like bandits as huge percentages of scalpers start blowing up. The dealing desk earns exactly as much as the destroyed scalpers lost.

Despite the long term advantage, the desks hate watching the equity drain slowly everyday. The common tactics result in price shading, poor execution and dealer intervention where the dealing desk outright refuses to close the trader’s position. If you like scalping the markets, trading on a dealing desk is not a good idea.

Traders with average holding times in excess of four hours might consider trading with a dealing desk given an appropriate incentive. FXCM now offers a fairly compelling reason to seriously consider the idea. Despite all the breathless criticism of FXCM across the internet, my primary gripe with the company is that they charge an outrageous amount of money. Their spreads are consistently higher than every other major brokerage. I consistently see them charging 2.6 pips on the EURUSD, yet their interbank feeds run average spreads around 0.5 pips. Charging a 400%+ markup is outrageous.

The new dealing desk offering reduces the lowest possible spread to 1.5 pips. That one pip of difference takes FXCM from being one of the most overpriced in the market to offering one of the lower spreads available among major brokerages. The spread pricing is far more reasonable and offers the opportunity to trade with a reputable company.

I am comfortable endorsing the idea of trading on FXCM’s dealing desk so long as the traders holds his positions for at least several hours. Although it comes with obvious disadvantages and conflicts of interest, those factors are much less likely to apply to a trader the longer that he holds a position. Carry traders and those with multi-month holding times have the least reason to be concerned. Trading with a dealing desk in exchange for a 40% discount in trading costs is worth it.

Scalpers should avoid the setup. No dealing desk wants scalpers on its system. I’m told that FXCM will push scalpers who trade on the dealing desk automatically onto pass through execution, which FXCM refers to as NDD (no dealing desk). Scalpers can maintain the dealing desk pricing but are secretly switched onto FXCM’s NDD feed.

Nonetheless, I feel that the conflict of interest is to great for this category and that traders should not rely on the company’s unwritten policies for decent execution. Scalpers at FXCM are better off trading on the NDD feed. Better yet, scalpers should find a broker with pass through execution that charges reasonable spreads.

If you trade six figure balances and/or do a lot of trading volume (more than 100 million notional) and are concerned about trading costs, please contact me directly at info@onestepremoved.com. OneStepRemoved.com is not an introducing broker and does not receive rebates on its referrals. I am happy, however, to introduce readers to companies and introducing brokers that offer trading costs under 1 pip and without any spread markups.

Convert to MetaTrader 5

by 2 Comments

Alpari’s recent launch of MetaTrader 5 triggered a small wave of MQL5 translation requests. Most traders assume that MT5 is about to take over the world. Perhaps it’s better to front run any potential problems. I assure you, though, that there’s no need to panic.

The launch mainly signifies that the larger forex firms will start rolling out their own installations of MT5 within the next six to twelve months. Rumor has it that Alpari’s owners are very close to the owners of MetaQuotes. Perhaps this is hearsay, but it’s my impression that Alpari is the first among equals when it comes to MetaQuotes’ clientele. Alpari did pay up the wazoo, however, for their license. Maybe they’re just getting rewarded for adopting the new platform so quickly.

Most brokerages, especially the large ones, are not chomping at the bit to adopt the new release. In fact, most of them hate MetaTrader with a passion. The back office is written largely for brokerages that exclusively want to use MetaTrader. The larger brokers, all of whom invariably offer their own proprietary platforms, have to jump through a lot of hoops to get all the moving parts between separate back office systems working in sync. The rollout will likely embroil their IT staff in problems for months on end. I seriously doubt most CEOs are looking forward to the switch.

Also, offering MT5 as the primary platform does not mean that your brokerage is going to flip the off switch on MT4. They depend on MetaTrader 4 for their cash flow. Brokerages will not sabotage themselves by preventing all of their customers from trading.

Rollouts of new technologies usually occur over a period of 9 months or more. When I worked with FXCM, I remember the handful of clients that refused to switch from Trading Station I to II. It wasn’t until 2 years after the initial release of the new version where the company decided to drag the stragglers kicking and screaming onto version II.

The switch from MetaTrader 3 to 4 worked in much the same way at the brokerages offering it at the time. It wasn’t until two years or so after its initial adoption that version 3 went by the wayside.

You have little to worry about as a retail trader considering the switch over to MT5. If you want to program a brand new EA and your broker already supports MetaTrader 5, then you should definitely program it in MQL5. Otherwise, stick with MetaTrader 4. It still has years of shelf life.