Algorithmic and Mechanical Forex Strategies | OneStepRemoved

How badly do I want in?

by 10 Comments

You absolutely must check your trading system’s performance on a regular basis. You’re going to miss most of the problems from watching your equity curve alone.

That almost happened to me a few weeks ago. When I observed my account, I noticed that the real results had dramatically underperformed the hypothetical results. A quick review showed me that I only took 271 trades over the prior week, whereas my backtest expected to find 360.

I was only trading 75% of the setups! What could explain the missing trades?

Finding the flaw

One feature that I wrote into the MetaTrader version of the Dominari was a maximum spread feature. I’m paying commissions, so the idea of the rare but possible scenario of paying a 10 pip spread to enter a trade seemed intolerable. I added a maximum spread feature to prevent getting ripped off.

I also didn’t put much thought into what happens if the spread is too wide. My initial instinct was to put the EA into hibernation for a few seconds. It would then wake up and check the spread. If the spread narrowed enough, it would send a market order. But in my haste to start trading, I forgot to also require that the price be near my original requested price. That design would have allowed the market to drift up 10 pips and then, if the spread narrowed, dramatically overpay to get in the trade.

The new method for capping the spread paid uses limit orders if the spread is too wide. The advantage to this method is that it solves two simultaneous problems. The first one is easy to understand. A limit order has a limited price. It’s not possible for the price drift described in the above paragraph to occur. I either get the price I want or the market moves without me and I miss the trade.

Equity curve since I made the execution changes on March 16.

Equity curve since I made the execution changes on March 16.

The second advantage to using limit orders on entry is the fact that a limit order rests on the broker’s server. The hibernating method could potentially miss fractions of a second where the spread temporarily narrows to an acceptable price. Limit orders catch all price quotes, improving my theoretical likelihood of a fill.

Reality proved the theory after a week of trading. Instead of taking 75% of all possible signals, I’m now taking 87.5% of signals. That’s a result of the new limit method and my willingness to pay a wider spread to enter a trade.

More improvement

The question at the top of my mind was, “Should I be willing to pay even more to enter these trades?” Like a good quant, I immediately decided to calculate the question instead of haphazardly guessing.

I wrote a script in MetaTrader to search for every limit order in my account which was cancelled. I then looked at what the hypothetical performance of those trades would have been if I had simply paid the exorbitant spread.

It turns out that I should be willing to pay a lot more money to enter these trades.

There have been 50 cancelled limit orders within the past week, 44 of which were theoretically profitable. The average theoretical profit per trade was €1.28 compared to €0.33 for all executed trades. That’s a massive 287% difference in profitability!

The other shocker was the percent accuracy. 44 out of 50 implies an accuracy of 88%, compared to 64% accuracy on executed trades. 50 signals isn’t a lot. Am I getting too excited about missed profits or is that bad luck?

Basic statistics gives an answer with a high degree of precision. If the real accuracy is 64%, then you would expect to see 50 * 0.64 = 32 winning trades in a random sampling. My observed, theoretical accuracy with these limit orders was 44 orders out of 50, which is 88% accurate.

It turns out that I should be willing to pay a lot more money to enter these trades.

The standard deviation for 64% accuracy on 50 orders is 0.48, which we can then use to calculate the standard error. The standard error on 50 orders is sqrt(50) * 0.48 = 3.42 orders.

And finally, the standard error gives us enough information to compute the z-score. The z-score is the observed values-expected values/standard error, which is (44-32) / 3.42 = 3.5. A z-score of 3.5 has a probability of 0.000233 occurring due to random chance, or about 1 in 4,299 tests.

Conclusion: The statistics say with high confidence that my non-executed orders are substantially more accurate than my executed orders.

With the orders being both more accurate and having a higher per trade value, I increased the maximum spread that I’m willing to pay by 53%. While that sounds oddly precise, the per trade value might be substantially overestimated. I ball parked a guess that paying 40% in trade costs for a high quality trade seems reasonable. That number may have to go higher in order for me to measure the details.

Ideas for exploration

The amazing extrapolation from the live order analysis is that the spread seems to predict my likelihood of success. Wider spreads make me more likely to succeed and with a better risk:reward ratio. My project over the next few days will be to start logging my spreads at signal generation time to evaluate whether the spread predicts the profitability of my signals.

Oddly enough, there might even be a paradoxical outcome where narrow spreads predict my failure. More on that when I have enough data to answer the question.

Trading Lessons from the Latest Gold Spike

by 10 Comments

After a relatively long period of calm, Gold has come back from the dead. Half way through February, Gold prices have spiked by more than 12%. This, of course, has got some traders scratching their head, wondering where the heck the spike came from… as if Gold moves materialize out of thin air.

Well, Gold moves don’t just “happen.” In fact, the latest spike in Gold price provides some powerful trading lessons on trading Gold.

Charting Gold Volatility with MT4

Before we jump into the lessons to be learned let’s first focus on the cycles of Gold. That should help explain the latest Gold Spike. Despite what you may have heard, Gold does have cycles. Surprisingly, the cycles are not in the price itself but in the level of volatility. In the past, I expanded on the advanced use of derivatives to predict an upcoming Gold move and direction. But the lesson here is far simpler.

In the weekly Gold chart below, we have measured Gold’s volatility through Standard Deviation (or StdDev), which is available in MetaTrader.

As one can see, Gold’s StdDev is remarkably simple to analyze. Every time the StdDev falls to 20 it jumps back up. And every time Gold’s StdDev jumps to 80 it falls down, back to the 20 low. The only exceptions were two extreme cases when the StdDev reached 145.

Moreover, we can see that if StdDev has hit lows more than once the spike of volatility higher is almost certain.

Gold

Source: MT4

First Trading Lesson

Of course, our StdDev analysis is not unique to the latest spike. But what can we learn? The first interesting lesson is that volatility spikes tend to concentrate around resistance/support levels. In our case, that is the 1,050 support, aka Point A. That means that Gold opportunities tend to occur around support and resistance levels. Now, it may seem an obvious conclusion but here is what it actually means:

When you have a support or resistance level and StdDev is at 20 that’s the ideal time for entry. You will have both textbook support and resistance levels working smoothly.  Then you’ll know that a strong momentum is coming.

Second Lesson

The second lesson is a simple but important one. A spike in Gold volatility does not necessarily mean a break of support and resistance levels. When technicals suggest that the support level will hold look at the StdDev. If the StdDev is at 20 that’s good; it re-enforces that simple fact that the support level will hold.

Third Lesson

The third lesson, which combines the first two, is perhaps the most interesting. It is the understanding that one jump in StdDev is not equal to one move. Let me elaborate; say StdDev has hit the 20 low near the 1,050 support. Meanwhile, Gold prices have spiked but StdDev hasn’t yet reached the 80 high. That means that volatility is set to rise but it doesn’t necessarily mean that Gold will keep moving in the same direction.

Rather, it means it will just move with a strong momentum. In the case above we can see Gold has hit the upper price channel resistance at Point B. But StdDev suggests volatility still hasn’t been maximized. And that means that the rise in volatility, next time, could come in the form of a short.

What Did We Learn?

So if we take our three lessons and try to summarize them, what did we learn? We learned that when StdDev is at its 20 lows it’s the best time to trade Gold. That’s because it guarantees that you will get a strong price momentum and it validates your technical analysis on Gold.

The Ultimate Signal for Range Bound Trading

by 6 Comments

How does one identify that a range bound opportunity is descending upon us? We all know the trading signals for identifying trends and trend breaks, but how about those that identify a range bound? While most of us focus on riding the next trend that may not be where the next opportunity lies.

That might be in a range bound, where it’s easy to figure where the pair will move lower or higher. Of course, we all know what a range bound looks like in the charts. The problem is once we identify it as such, it might be too late. Many of the range bound trends become apparent only in retrospect, when the pattern crystallizes. But there is some good news. Just like tools that identify trends and trend breaks, there are effective tools for identifying the approach of a range bound.

Range Bound and Moving Standard Deviation

The single most evident quality of range bound is, quite simply, falling volatility or Standard Deviation. As Standard Deviation falls, the pair has smaller fluctuations and therefore is “bound” within a range. Likewise, when Standard Deviation is low we are stuck in a range.

If we can time Standard Deviation (and we can), we can know when it’s about to fall. If it is about to fall then we are heading to a range bound and can adjust our strategy accordingly. Moving Standard Deviation is a measure of the change of Standard Deviation through time. And it is exactly designed for our specific case and thus makes it the ultimate tool to time an upcoming range bound. Here’s how you do that:

Range Bound Trading

 

Source: e-signal

In the chart above, we have the Moving Standard Deviation or MSD running (at the bottom). It is evident that MSD tends to top out after surpassing the 80% level while it tends to bounce back after falling below 20%.

In part A and C, you can clearly see that as MSD topped out above 80% the pair became less volatile and then moved into a range. On the flip side, when MSD moved below 20%, we can expect the range bound to end and the trend to continue.

Validate the Range

First Validation: If the pair had been surging when the MSD surpassed the 80 level, that’s your high band of the range. On the other end, if the pair had been sliding when the 80 was reached, that’s your low. Whatever high or low was reached during the time the MSD hit above 80 that’s your first confirmation of the range you will be getting.

Second Validation: The confirmation of the other band of the range will present itself as a candle with a needle, rather shortly after.

A Few Rules of Thumb

MSD 20 – Exit Immediately: Regardless of direction in your range bound trade (long/short) or time elapsed since the 80 level signaled the range bound, be wary. If the MSD falls below 20 get ready to terminate your trade, because the trend is about to resume and your range might be over. Although it’s not always immediate, it’s better to take precaution rather than lose your gains by ignoring the below 20 signal.

MSD is Only Good for Range: That might sound tricky. As we just stated, the MSD will always fall below 20 before the trend resumes. You’ll know when to exit your range bound trade and be able to cap your risk of loss. But many times MSD will fall below 20 and the trend won’t immediately resume. Thus, don’t use the MSD to predict the opposite of a range bound; that is, to predict the trend. There are different tools for identifying when a trend is about to re-emerge.

Range Bound Cycles: Finally, as you can see in the chart, in the long run the cycles of range bound vs trending tend to be more or less the same length. That is always something to keep in mind. Although the MSD will be your mark to exit/enter the range, timing your cycles of range is always useful to let you know how much more time might be left in that lucrative range bound trend you are taking.

The Dimension of Time in Trading

by 6 Comments

The trading blogosphere is full of articles about the importance of timing your trade. Certainly, it’s important; identifying the right trend, knowing when to open or close – all integral to the trade. But here is the thing, there’s another dimension and timing is only half the story. You see, in the real world, it’s not just about the timing of your entry and exit but also the time between. In fact, the duration of your trade matters a lot more than you might think. In this article I’ll discuss the various aspects in which time can affect both short and long term strategies.

Duration and Returns

Let’s consider a theoretical trading situation, looking at two strategies. The first is a high frequency one designed at gaining 1 pip, with every trade taking a single second. We’ll call this Strategy A. The second strategy, Strategy B, is designed to generate a profit of 10 pips every 10 seconds. So which is the better strategy? On paper, both should generate 60 pips per minute hence equal returns. But in the real world? Well, that’s an entirely different story.

In real life, when you aggregate the results of the two strategies, you’re going to be in for a big surprise. Under Strategy A you had a profit of 50 pips after 1 minute. But under Strategy B you had a profit of 66 pips. Now which strategy do you think is better? Of course, you’d pick B. Interestingly, per trade, each strategy profited exactly as planned. Strategy A profited 1 pip and Strategy B profited 10 pips per trade.

So what made the difference? The time. As you can see below in the two charts, strategy A, when executed, took a little longer on average than initially expected. From the measurement of the duration of trades it clustered around 1.2 seconds per trade rather than 1 second. In Strategy B, the measurement of duration per trade, on average, is about 9 seconds. When executed, Strategy B trades took less time than initially expected. Eventually, at the end of the day, Strategy B returned 66 pips or 32% more than Strategy A’s 50 pips. That means Strategy B is the superior choice.

Now, let’s look at this a bit closer still. We’re going to take the duration of each of the trades executed and measure the standard deviation (a simple exercise in Excel). You would discover that in Strategy A, while the average might be 1 second per trade, the standard deviation is high. That suggests the cycles are not equal thus the strategy is less efficient. On the other hand, Strategy B has a low standard deviation, meaning most trades are close to 10 seconds. Therefore, it has more predictable returns and, hence, a more effective strategy. When you measure a strategy, understand that beyond returns and risk, the time each trade takes matters. And in fact, it can matter quite a lot.

Time in Trading

Time in trading

Duration and Swing Trading

So now we understand how the nuances of time can affect the eventual returns of each strategy. Yet when it comes to longer duration trades (i.e. weeks or months) there is still another element. In this case, it is the amount of time the trade has been positive.

Let’s look at another theoretical scenario. This time, we have Strategies C and D, each has the identical duration of two weeks. Likewise, both have the same standard deviation of duration and roughly the same returns. But one strategy is riskier than the other. The question is which? The answer is the strategy that has been profitable for the least amount of time. Say you opened two trades; Strategy C was profitable for 12 of the 14 days it took to reach its target. Strategy D had been negative for the first 12 days before producing steep returns in the last 2. Strategy D, then, is clearly more risky because you are risking a loss for a longer time. This also suggests that, perhaps, the entry signal for Strategy D is not very well calibrated. Strategy C is clearly more effective and less risky, on average.

Time in Trading

 

Time in Trading

The Bottom Line

Of course, there are many more aspects to measuring time effectiveness in a trading strategy. There are also other dimensions to examine beyond just time. But the lesson here is clear. The next time you measure your own trading strategy remember that time isn’t just about timing.

How To Win With Mechanical Trading Systems

by 13 Comments

Much ink has been devoted to pinpointing the causes of mechanical trading systems failures, especially after the fact. Although it may seem oxymoronic (or, to some traders, simply moronic), the main reason why these trading systems fail is because they rely too much on the hands-free, fire-and-forget nature of mechanical trading. Algorithms themselves lack the objective human oversight and intervention necessary to help systems evolve in step with changing market conditions.

Mechanical trading systems failure, or trader failure?

Instead of bemoaning a trading-system failure, it’s more constructive to consider the ways in which traders can have the best of both worlds:  That is, traders can enjoy the benefits of algorithm-managed mechanical trading systems, such as rapid-fire automatic executions and emotion-free trading decisions, while still leveraging their innate human capacity for objective thinking about failure and success.

The most important element of any trader is the human capability to evolve. Traders can change and adapt their trading systems in order to continue winning before losses become financially or emotionally devastating.

Choose the right type and amount of market data for testing

Successful traders use a system of repetitive rules to harvest gains from short-term inefficiencies in the market. For small, independent traders in the big world of securities and derivatives trading, where spreads are thin and competition fierce, the best opportunities for gains come from spotting market inefficiencies based on simple, easy-to-quantify data, then taking action as quickly as possible.

When a trader develops and operates mechanical trading systems based on historical data, he or she is hoping for future gains based on the idea that current marketplace inefficiencies will continue. If a trader chooses the wrong data set or uses the wrong parameters to qualify the data, precious opportunities may be lost. At the same time, once the inefficiency detected in historical data no longer exists, then the trading system fails. The reasons why it vanished are unimportant to the mechanical trader. Only the results matter.

mechanical trading rules

Pick the most pertinent data sets when choosing the data set from which to create and test mechanical trading systems. And, in order to test a sample large enough to confirm whether a trading rule works consistently under a wide range of market conditions, a trader must use the longest practical period of test data.

So, it seems appropriate to build mechanical trading systems based on both the longest-possible historical data set as well as the simplest set of design parameters. Robustness is generally considered the ability to withstand many types of market conditions. Robustness should be inherent in any system tested across a long time range of historical data and simple rules. Lengthy testing and basic rules should reflect the widest array of potential market conditions in the future.

All mechanical trading systems will eventually fail because historical data obviously does not contain all future events. Any system built on historical data will eventually encounter ahistorical conditions. Human insight and intervention prevents automated strategies from running off the rails. The folks at Knight Capital know something about live trading snafus.

Simplicity wins by its adaptability

Successful mechanical trading systems are like living, breathing organisms. The world’s geologic strata are filled with fossils of organisms which, although ideally suited for short-term success during their own historical periods, were too specialized for long-term survival and adaptation. Simple algorithmic mechanical trading systems with human guidance are best because they can undergo quick, easy evolution and adaptation to the changing conditions in the environment (read marketplace).

Simple trading rules reduce the potential impact of data-mining bias. Bias from data mining is problematic because it may overstate how well a historical rule will apply under future conditions, especially when mechanical trading systems are focused on short time frames. Simple and robust mechanical trading systems shouldn’t by affected by the time frames used for testing purposes. – The number of test points found within a given range of historical data should still be large enough to prove or disprove the validity of the trading rules being tested. Stated differently, simple, robust mechanical trading systems will outshine data-mining bias.

If a trader uses a system with simple design parameters, such as the QuantBar system, and tests it by using the longest appropriate historical time period, then the only other important tasks will be to stick to the discipline of trading the system and monitoring its results going forward. Observation enables evolution.

On the other hand, traders who use mechanical trading systems built from a complex set of multiple parameters run the risk of “pre-evolving” their systems into early extinction.

Build a robust system that leverages the best of mechanical trading, without falling prey to its weaknesses

It’s important not to confuse the robustness of mechanical trading systems with their adaptability. Systems developed based on a multitude of parameters led to winning trades during historical periods – and even during current observed periods – are often described as ‘robust.’ That is no a guarantee that such systems can be successfully tweaked once they have been trade past their “honeymoon period.” That is an initial trading period during which conditions happen to coincide with a certain historical period upon which the system was based.

Simple mechanical trading systems are easily adapted to new conditions, even when the root causes of marketplace change remain unclear, and complex systems fall short. When market conditions change, as they continually do, the trading systems which are most likely to continue to win are those which are simple and most-easily adaptable to new conditions; a truly robust system is one which has longevity above all.

Simple algorithmic mechanical trading systems with human guidance are best because they can undergo quick, easy evolution and adaptation to the changing conditions in the environment (read marketplace).

Unfortunately, after experiencing an initial period of gains when using overly-complex mechanical trading systems, many traders fall into the trap of attempting to tweak those systems back to success. The market’s unknown, yet changing, conditions may have already doomed that entire species of mechanical trading systems to extinction. Again, simplicity and adaptability to changing conditions offer the best hope for survival of any trading system.

Use an objective measurement to distinguish between success and failure

A trader’s most-common downfall is a psychological attachment to his or her trading system. When trading-system failures occur, it’s usually because traders have adopted a subjective rather than objective viewpoint, especially with regard to stop-losses during particular trades.

Human nature often drives a trader to develop an emotional attachment to a particular system, especially when the trader has invested a significant amount of time and money into mechanical trading systems with many complex parts which are difficult to understand. However, it’s critically important for a trader to step outside the system in order to consider it objectively.

In some cases, the trader becomes delusional about the expected success of a system, even to the point of continuing to trade an obviously-losing system far longer than a subjective analysis would have allowed. Or, after a period of fat wins, a trader may become “married” to a formerly-winning system even while its beauty fades under the pressure of losses. Worse, a trader may fall into the trap of selectively choosing the testing periods or statistical parameters for an already-losing system, in order to maintain false hope for the system’s continuing value.

An objective yardstick, such as using standard deviation methods to assess the probability of current failure, is the only winning method for determining whether mechanical trading systems have truly failed. Through an objective eye, it’s easy for a trader to quickly spot failure or potential failure in mechanical trading systems, and a simple system may be quickly and easily adapted to create a freshly-winning system once again.

Failure of mechanical trading systems is often quantified based on a comparison of the current losses when measured against the historical losses or drawdowns. Such an analysis may lead to a subjective, incorrect conclusion. Maximum drawdown is often used as the threshold metric by which a trader will abandon a system. Without considering the manner by which the system reached that drawdown level, or the length of time required to reach that level, a trader should not conclude that the system is a loser based on drawdown alone.

Standard deviation versus drawdown as a metric of failure

In fact, the best method to avoid discarding a winning system is to use an objective measurement standard to determine the current or recent distribution of returns from the system obtained while actually trading it. Compare that measurement against the historical distribution of returns calculated from back-testing, while assigning a fixed threshold value according to the certainty that the current “losing” distribution of mechanical trading systems is indeed beyond normal, to-be-expected losses, and should therefore be discarded as failed.

So, for example, assume that a trader ignores the current drawdown level which has signaled a problem and triggered his investigation. Instead, compare the current losing streak against the historical losses which would have occurred while trading that system during historical test periods. Depending upon how conservative a trader is, he or she may discover that the current or recent loss is beyond, say, the 95% certainty level implied by two standard deviations from the “normal” historical loss level. This would certainly be a strong statistical sign that the system is performing poorly, and has therefore failed. In contrast, a different trader with greater appetite for risk may objectively decide that three standard deviations from the norm (i.e. 99.7%) is the appropriate certainty level for judging a trading system as “failed.”

The most important factor for any trading systems’ success, whether manual or mechanical, is always the human decision-making ability. The value of good mechanical trading systems is that, like all good machines, they minimize human weaknesses and empower achievements far beyond those attainable through manual methods. Yet, when properly built, they still allow firm control according to the trader’s judgment and allow him or her to steer clear of obstacles and potential failures.

Although a trader can use math in the form of a statistical calculation of standard distribution to assess whether a loss is normal and acceptable according to historical records, he or she is still relying on human judgment instead of making purely-mechanical, math-based decisions based on algorithms alone.

Traders can enjoy the best of both worlds. The power of algorithms and mechanical trading minimizes the effects of human emotion and tardiness on order placement and execution, especially with regard to maintaining stop-loss discipline. It still uses the objective assessment of standard deviation in order to retain human control over the trading system.

Be prepared for change, and be prepared to change the trading system

Along with the objectivity to detect when mechanical trading systems change from winners into losers, a trader must also have the discipline and foresight to evolve and change the systems so they can continue to win during new market conditions. In any environment filled with change, the simpler the system, the quicker and easier its evolution will be. If a complex strategy fails, it may be easier to replace than to modify it, while some of the simplest and most-intuitive systems, such as the QuantBar system, are relatively easy to modify on-the-fly in order to adapt to future market conditions.

In summary, it can be said properly-built mechanical trading systems should be simple and adaptable, and tested according to the right type and amount of data so that they will be robust enough to produce gains under a wide variety of market conditions. And, a winning system must be judged by the appropriate metric of success. Instead of merely relying on algorithmic trading rules or maximum drawdown levels, any decision about whether a system has failed should be made according to the trader’s human judgment, and based on an assessment of the number of standard deviations of the system’s current performance when measured against its historic-test losses. If mechanical trading systems are failing to perform, the trader should make the necessary changes instead of clinging to a losing system.