Algorithmic and Mechanical Forex Strategies | OneStepRemoved

  • Articles
  • Sophisticated Web Sites
  • Automated Trading
  • Testimonials
  • Contact

How badly do I want in?

March 22, 2016 by Shaun Overton 10 Comments

You absolutely must check your trading system’s performance on a regular basis. You’re going to miss most of the problems from watching your equity curve alone.

That almost happened to me a few weeks ago. When I observed my account, I noticed that the real results had dramatically underperformed the hypothetical results. A quick review showed me that I only took 271 trades over the prior week, whereas my backtest expected to find 360.

I was only trading 75% of the setups! What could explain the missing trades?

Finding the flaw

One feature that I wrote into the MetaTrader version of the Dominari was a maximum spread feature. I’m paying commissions, so the idea of the rare but possible scenario of paying a 10 pip spread to enter a trade seemed intolerable. I added a maximum spread feature to prevent getting ripped off.

I also didn’t put much thought into what happens if the spread is too wide. My initial instinct was to put the EA into hibernation for a few seconds. It would then wake up and check the spread. If the spread narrowed enough, it would send a market order. But in my haste to start trading, I forgot to also require that the price be near my original requested price. That design would have allowed the market to drift up 10 pips and then, if the spread narrowed, dramatically overpay to get in the trade.

The new method for capping the spread paid uses limit orders if the spread is too wide. The advantage to this method is that it solves two simultaneous problems. The first one is easy to understand. A limit order has a limited price. It’s not possible for the price drift described in the above paragraph to occur. I either get the price I want or the market moves without me and I miss the trade.

Equity curve since I made the execution changes on March 16.

Equity curve since I made the execution changes on March 16.

The second advantage to using limit orders on entry is the fact that a limit order rests on the broker’s server. The hibernating method could potentially miss fractions of a second where the spread temporarily narrows to an acceptable price. Limit orders catch all price quotes, improving my theoretical likelihood of a fill.

Reality proved the theory after a week of trading. Instead of taking 75% of all possible signals, I’m now taking 87.5% of signals. That’s a result of the new limit method and my willingness to pay a wider spread to enter a trade.

More improvement

The question at the top of my mind was, “Should I be willing to pay even more to enter these trades?” Like a good quant, I immediately decided to calculate the question instead of haphazardly guessing.

I wrote a script in MetaTrader to search for every limit order in my account which was cancelled. I then looked at what the hypothetical performance of those trades would have been if I had simply paid the exorbitant spread.

It turns out that I should be willing to pay a lot more money to enter these trades.

There have been 50 cancelled limit orders within the past week, 44 of which were theoretically profitable. The average theoretical profit per trade was €1.28 compared to €0.33 for all executed trades. That’s a massive 287% difference in profitability!

The other shocker was the percent accuracy. 44 out of 50 implies an accuracy of 88%, compared to 64% accuracy on executed trades. 50 signals isn’t a lot. Am I getting too excited about missed profits or is that bad luck?

Basic statistics gives an answer with a high degree of precision. If the real accuracy is 64%, then you would expect to see 50 * 0.64 = 32 winning trades in a random sampling. My observed, theoretical accuracy with these limit orders was 44 orders out of 50, which is 88% accurate.

It turns out that I should be willing to pay a lot more money to enter these trades.

The standard deviation for 64% accuracy on 50 orders is 0.48, which we can then use to calculate the standard error. The standard error on 50 orders is sqrt(50) * 0.48 = 3.42 orders.

And finally, the standard error gives us enough information to compute the z-score. The z-score is the observed values-expected values/standard error, which is (44-32) / 3.42 = 3.5. A z-score of 3.5 has a probability of 0.000233 occurring due to random chance, or about 1 in 4,299 tests.

Conclusion: The statistics say with high confidence that my non-executed orders are substantially more accurate than my executed orders.

With the orders being both more accurate and having a higher per trade value, I increased the maximum spread that I’m willing to pay by 53%. While that sounds oddly precise, the per trade value might be substantially overestimated. I ball parked a guess that paying 40% in trade costs for a high quality trade seems reasonable. That number may have to go higher in order for me to measure the details.

Ideas for exploration

The amazing extrapolation from the live order analysis is that the spread seems to predict my likelihood of success. Wider spreads make me more likely to succeed and with a better risk:reward ratio. My project over the next few days will be to start logging my spreads at signal generation time to evaluate whether the spread predicts the profitability of my signals.

Oddly enough, there might even be a paradoxical outcome where narrow spreads predict my failure. More on that when I have enough data to answer the question.

Filed Under: Dominari Tagged With: execution, limit, quant, slippage, standard deviation, standard error, statistics, Z-score

Comments

  1. MITCHELL says

    March 24, 2016 at 16:02

    I MAY NOT BE ASKING THE RIGHT QUESTION, HOWEVER, WHAT WOULD THE HIGHER SPREAD DO TO THE 75% EXECUTED TRADE PROFIT/LOSS? WELL WRITTEN AND THANKS.

    Reply
    • Shaun Overton says

      March 24, 2016 at 16:14

      My limit orders stipulate the WORST price that I’m willing to pay, and I only send them out when the spread is too wide.

      Say that I’m watching EURUSD where the prices are 1.08500 bid and 1.08505 ask. If I was only willing to pay a 0.3 pip spread on EURUSD and I wanted to buy, then I’d set my limit order to 1.08500 (the bid) + 0.00003 = 1.08503.

      Buy orders only use the ask. The current ask would have to decline from 1.08505 to 1.08503 in order for my trade to enter the market.

      Reply
  2. 4 sure says

    March 24, 2016 at 16:10

    This is all very interesting and thanks for sharing. But, all of this is based on such a small sample of data it really could easily fall into the category of random chance.

    So in answer to the question: “Am I getting too excited about missed profits or is that bad luck?” I’d say yes and no.

    I do realize these types of issues are critically important for the kind of high volume, small target trading system you’re trying to employ. Which surprises me that commissions, spreads and the like weren’t REALLY thoroughly researched before launching some iterations ago especially after some—after reviewing your projected theoretical results—issued comments and warnings regarding the broker take with this kind of strategy.

    Anyway, thanks again for sharing. Very informative.

    Reply
    • Shaun Overton says

      March 24, 2016 at 16:18

      But, all of this is based on such a small sample of data it really could easily fall into the category of random chance.

      That’s the point of doing the article. No, it most definitely is not the result of random chance. Such an occurrence would only happen once in every 4,299 trials (99.97% certainty that it’s not random).

      Secondly, you can’t thoroughly research broker costs in advance. You have to push volume through their systems to get real numbers. Anything less than that is just pie in the sky, made up guesses. It’s better to spend some money and get real information instead of speculating.

      Reply
  3. Derek Phelps says

    March 24, 2016 at 19:41

    Interesting results.
    With an 88% profitable on just the trades you previously rejected vs 64% with the trades you did take along with a much higher return per trade maybe consider this for the coming week or two…
    Trade your strategy unchanged along with a second identical strategy variant that ONLY takes the trades the first one would reject as ‘too expensive’ for a comparative analyses of the results. Trading the same account you would still be able to show the net P&L curve, but could easily differentiate between the expected profitability of the two methods. If your lower frequency but higher expected PPT results maintain a high correlation to your previous sample then you could consider increasing the entry size by a factored amount on the second strategy (bang for buck)

    Reply
    • Shaun Overton says

      March 25, 2016 at 08:55

      Hey Derek,

      Good to hear from you again. I’m glad to see that you’re still lurking around the site.

      I could do that, but there are practical considerations. The main one is that the portfolio size is already at the bare minimum. Anything less than $10,000 distorts the results due to rounding the lot sizes. I’m going to accomplish the same thing by recording spreads at the time a signal generates. If there’s a clear relationship between more spread = more profit, I’ll then have a solid, mathematical foundation for whatever course of action that I take next.

      Reply
  4. Sandro Mechanical Trader says

    March 25, 2016 at 08:17

    hello, Shaun Overton, congratulations for your blog.
    I also had this problem on my system.
    in backtest results are better, but in actual operations.
    are not the same as in the backtest, fixed spreads are configured, even a maximum;
    but the reality is different. also use pending orders.
    which realized and concludes, it is that the brokers to see any sign of great volatility, almentam their spreads, but why? it is only because the volatility increased, I think not. the question is who with high volatility and rapid price, if you have a better probability of success on the part of traders, as the rapid price is much easier to win. and going in the right direction.
    in this case the mechanical systems are at a disadvantage if you set a maximum spread value, I’m in the laboratory to look at the question nomeu mechanical system.
    but I think a good logic test, it would be a dynamic spread limit depending on market conditions, creating an adaptive heuristic, if volatility increases, and in times of economic announcements., is just an idea

    Reply
    • Shaun Overton says

      March 25, 2016 at 08:57

      Thanks, Sandro.

      I agree that spreads reflect the underlying market condition. It’s not an idea that I would’ve arrived at from a thought experiment. I would’ve viewed wider spreads as a result of wider volatility instead of a profit indicator. I’m still not convinced that spreads will predict my system profitability, but there’s enough evidence for me to spend the time ruling it out.

      Reply
  5. Andrew says

    February 25, 2018 at 19:16

    In your discussion of Z score

    “And, positive Z indicates that the profitable trade will be followed by another profitable one, and a loser will be followed by another loss. This observed dependency lets the forex trader vary the position sizes for individual trades in order to help manage risk.”

    That should be negative Z I think.

    Reply
    • Shaun Overton says

      February 26, 2018 at 16:25

      Where does it say that in the article?

      Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

FREE trading strategies by email

Trending

Sorry. No data so far.

Archives

  • Dominari
  • How does the forex market work?
  • Indicators
  • MetaTrader Tips
  • MQL (for nerds)
  • NinjaTrader Tips
  • Pilum
  • QB Pro
  • Stop losing money
  • Test your concepts historically
  • Trading strategy ideas
  • Uncategorized
  • What's happening in the current markets?

Translation


Free Trading Strategies

Privacy PolicyRisk Disclosure

Copyright © 2023 OneStepRemoved.com, Inc. All Rights Reserved.