あなたは戦略の統計データのエントリ効率と出口効率をテストしていない場合はあなたの外国為替backtestsは絶対に価値がないです. 必然的にバックテストを実行している誰もが、結果として得たドルを報告. 他の要因には、損失の平均の勝利のような存在します, 利益率とシャープレシオ, しかし、彼らは、自動取引システムの設計の最終段階まで、あなたに有益な何かを教えていません.
戦略をテストへの正しいアプローチは、問題に焦点を当てるべきです, “私の戦略は、ごみの作品です?” ほとんどの人は、右の自分自身を証明しようとします. 本当のテストは、自分が間違っていることを証明することはできないことです. それを行うための唯一の方法は、統計的手法を介して行われ.
入口と出口の効率性
効率は、利用可能な取引レンジの何パーセントその戦略のキャプチャにハード番号を置きます. 取引ウィンドウは貿易が市場に参入したバーで開始. 時貿易の終了ウィンドウが閉じます.
利用可能な全ウィンドウは最高のマイナスウィンドウで最も低い低いです. 入口と出口の効率を計算することは、単にあなたの戦略をキャプチャする傾向がそのウィンドウの何パーセントを測定. 取引のすべての平均を取ると、あなたは全体の効率を得ます.
エントリ効率式
長い貿易のための式: (最高高 – 入口価格) ÷ (最高高 – 最低低)
短い貿易のための式: (入口価格 – 最低低) ÷ (最高高 – 最低低)
終了効率式
長い貿易のための式: (出口価格 – 最低低) ÷ (最高高 – 最低低)
短い貿易のための式: (最高高 – 出口価格) ÷ (最高高 – 最低低)
あなたがで架空の通貨を買う例を見てください 150 そして、でそれを販売 170. 入口と出口の時間と最低の低でした 140. The price then ran all the way up to 200 before settling back down to 170, which is where the exit took place.
The entry efficiency is (200-150) ÷ (200-140) = 50 ÷ 60 = 83%. Nearly anyone would agree that this makes for a great entry.
The exit efficiency is (170-140) ÷ (200-140) = 30 ÷ 60 = 50%. Most would agree that the exit would have ideally occurred sooner than it did.
Efficiencies do not change by instrument or time frame
One major problem that we encounter with forex backtests is the limited data set. This is especially true for those interested in testing long term strategies like those on the H4 or D1 charts. The wonderful thing about entry and exit efficiencies is that they do not vary from chart to chart or even period to period.
I like to jump down to M1 charts for efficiency testing. The data is nearly endless. 私は不足して心配する必要はありません. 私はH4チャートに戻って移行する際の素晴らしいところは、私が知っているということです, 効率は以上±5%を変更しないでください.
あなたが表示された場合、効率はあまり変化させます, あなたは統計的に有意なグループを形成するために十分な取引を持っていない可能性があり. 私の経験では、と言われます 75 取引は通常、実際の効率に非常に近い取得します. 100 取引以上が良いです. 私は、M1チャート上でテストを実行すると、, 私は、多くの場合、数ヶ月にわたって数千の取引を取得します. 大戦略のパラメータが本当にどれだけ強固な信頼の多大であなたを伝えることができることを数値.
通常, あなたがとることのできる範囲内にすべての結果 45-55% ランダムの結果であります, 確率過程. 私は右のようなものの障壁に忍び寄るbacktestsを見ると 54.9% あるいは 55.1%, バックの周りに結果必然的にタンク 50% マーク.
ランダム貿易の結果とドルの利益
私は、このセクションでは、ランダムな効率でお金を稼ぐ方法についてでした望みます. 悲しいかな, 我々は、ランダム性が不当eurphoriaをもたらすことができる方法をカバーしなければなりません.
私は今、数年前から乱数の概念に興味を持っていました. 数学者はより不透明な名前でそれを参照してください。 “確率過程”. 無意味名前にもかかわらず、, それはランダムの研究を言ってだけの空想の方法です – それはどのように変化します, その分布, どこまでそれ “あるきます”, など.
昨日, 私はコインのアナロジーがどのように記述するために使用されるフリップ マーチンゲール戦略 確率的に失敗する運命にされています. 私は言及しなかった一つの興味深い概念はブラウン運動に関係します. でもランダムな結果のセットで, 取引は出発点から離れてランダムウォークに行きます.
アインシュタインは、コンセプトの背後に数学を解決するための本当のクレジットを取得, 彼の名前は、用語にないにもかかわらず、. 彼はランダムなプロセスが続く距離は試行回数の平方根であることを実証しました. 我々は、コインを反転することを決定した場合 60 回, 私達はことを知っています 50% 時間の頭の上に落下し、他の必要があります 30 尾の.
それは実際に私たちが勝者か敗者のいずれかの数が非常にわずかなバイアスを期待するべきであることが判明, 我々はどちらかわからないが、. それはランダムです. 正確なバイアス, どちらの方法それは行くことを好みます, 等しくなければなりません √60, これはうまくいく〜7. ヘッドの成果は、一般的にからの範囲であるべきです 23-37, 尾の結果が違いを構成すると.
60のうち七取引は強く割合を変化させます, 我々は、それが本当にことになっていることを知っていても 50%. ヘッドが付属している場合のみアップ 23 times out of 60, それです 38%. The problem is not with the coin. It’s with the number of trials. As you do an increasing large number of trails, the random bias decreases in significance in terms of the percent accuracy. 50,000 取引, たとえば, should show a surplus of roughly 223 trades in favor of winning or losing. The accuracy range falls within 1% の 50% on either side, a dramatic improvement.
Risks of curve fitting
Curve fitting a random efficiency relates to the idea of Brownian motion. Let’s say that we use a strategy that I know will never show an entry or exit efficiency: the moving average crossover. I’ve gone through this strategy six ways from Sunday, almost exclusively at the behest of clients. It does not work as a fully automated strategy. There is no secret set of fast and slow periods that will unlock the hidden keys to profit.
Most traders, experienced or not, abuse the backtester by searching for a set of parameters that yield the most dollar profit. They curve fit their test to optimize for maximum profit. What really happens is that the traders optimize for amount of random drift that already occurred.
When I used the example of 50,000 trades creating a natural drift of 223, I cited it with the purpose of showing how little it reduces the error in the real percent accurracy. The other consequence for trading systems is that as the error percentage 減少, the natural bias in your outcomes 増加. Blindly running the optimizer only selects the set of combination that yields a combination of two criteria:
- The drift that happened to work out in favor of that set of parameters
- The profit and loss that varies with those parameters. The dollar profit naturally changes because the two moving averages cross at different points
You need a tool like efficiency to guard against these types of random outcomes. It’s the only method that I know of that definitively states whether or not a strategy behaves in a random manner. I especially like the fact that it breaks those elements down into two of the three basic components of a trading strategy: the entry, the exit, and the position sizing.
Efficient strategies do not work all of the time
Position sizing marks the final obstacle to building your fully automated trading strategy. A set of rules that yields a statistically efficient entry that is paired with an efficient exit does not necessarily make money. The value of each trading setup can vary, あまりにも.
Each strategy contains different sets of winners and losers. Each winner and loser varies in its dollar value. Whatever money management approach that you take requires balancing the ratio of the winners and losers in a way that normalizes the outcome of each trade. You ideally want to eliminate the variation in dollar value. 20 pip trades should earn or lose you exactly as much as the 100 pip trades.
That seems counter-intuitive. Most traders want to win in proportion with the size of the opportunity. It’s better from a system perspective to entirely ignore the size of the opportunity and to make each trade worth the same amount. Betting more or less with each trade effectively normalizes the value of each trade.
Using a stop loss stands out as an obvious candidate to fix how much a trade is worth. The severe disadvantage is that it almost always negatively affects the exit efficiency. Whenever I can get away with it, I always recommend using a market based exit instead of an arbitrary stop loss. Traders usually scream at the top of their lungs when they hear me say this. I’m just speaking as a systems developer. The numbers are what they are.
I liked this article. The paragraph about “Risks of curve fitting” is interesting… It is also called in Ernie Chan book (Quantitative Trading: How to Build Your Own Algorithmic Trading Business) “data-snooping biases” that’s why he advises to have two parts for your data “in sample data” (to optimize settings) と “out sample data” (to test settings)
The concept of testing “effectiveness” または “efficiency” of entry/exit point is also interesting… that’s an other point of view that I will probably try to consider in my next backtests.
I ever see an idea that is not present in your article that is called “walk forward analysis” WFA or “walk forward optimization” WFO… a picture is sometimes better than text… https://www.google.fr/search?q=walk+forward+optimization
What is your opinion about this way of backtesting / optimizing ?
Hi Femto,
あなたのコメントのおかげで. I highly recommend Ernie Chan’s book. It sits proudly on my bookshelf.
Walk forward analysis is exactly the same idea that you outlined from Chan with data-snooping bias. Walk forward is the more commonly accepted terminology. As for my opinion, walk forward is the only acceptable method for strategy testing.
I know it might seems silly or dumb,
As we know, backtesting in walkforward test is to check if a system robust ennough to trade in the out of sample data.
Lets say we do optimized in insample data 2000-2008, then test it on out sample data at 2008-2013. If the strategy still generated profit in 2008-2013 outsample data, so the system is robust ennough.
Now how about if we optimized the data in 2008-2013 as in sample data, then running test it at 2000-2013. I’ve seen some of my test generated profit also in those back out sample. So we could see, the system is robbust also for the market condition in the past that we didnt optimized it.
The thing is, we have optimized data with closest to current market condition. Som its had more adaptability to current market condition with the prove of robustnest in out sample market data in the past.
I guess The point of walkforward is to had the robustness as out sample data that not been optimized. So when the system had the same robustness with those out sample in the backward test, why dont we take it?
And the another point is not, where we will trade our real koney in the future, but actually wr trade our real money in the “out sample/ unknown data/market condition”.
Anything that we didnt optimized is an “out sample/unknown market condition”. So i guess , walk backward test ( i dont know what to name this idea ) had fullfill those out sample robustness if it make profit also in the out sample past. Of course with benefit of more adaptability to current market situation since the optimsation process is the nearest with current market.
Might be doing the optimisation every monthly, or if there was some basic aspect like historical drawdown breached, the pn we re-optimzed again for the past 2-3 years and also check for robustness of this optimization set to the backward out sample.
Is this thing make sense statistically? Please let me know you oppinion
Best Regards
Odie
Hey Odie,
There’s no mathematical theorem that I can give to disprove the idea. What I feel strongly about is that it doesn’t work.
Let’s say you optimize for 2008-2010, その後、 “forward test” the idea from 2000-2007. To some extent, the idea is already known and studied for the time that you’re testing.
I actually spent the last month experiencing this very idea. A client tasked me with recreating his manual trading strategy, based on data supplied from 2010. I tested my algorithm, which did a fair job of approximating his trades. I was curious about prior performance, so I ran a test all the way back to 2000. The results, not surprisingly, were equally solid. If I stopped there, it’d look like 10 years of solid performance optimized only on a single year.
その後、, along came the dreaded walk forward test. The results completely fell apart. “Forward testing” on previous data didn’t accomplish anything.
Thanks for the great question!
こんにちはショーン,
Congrats on your website. Lots of solid information here. I’ve just read with interest your article discussing the concept of measuring entry and exit efficiency. You’ve provided the formulas, but you don’t explain how to apply them. I’m in the process of backtesting a number of strategies in MT4. Is it possible to code the formulas into the EA I’m using for backtesting?
I look forward to hearing back from you.
drofwarc
Hi drofwarc,
I’ve honestly moved on from efficiency to different analysis tools. Efficiency is still solid for throwing out most unadvantaged entry and exit strategies. ということで, it doesn’t mean that what’s left is worth keeping.
It’s not worth building the efficiency measurements when you can program a strategy quickly in TradeStation or NinjaTrader to uncover the same number.
–ショーン