curve fitting Archives - アルゴリズムと機械の外国為替戦略

2 Painful hits

2 月 13, 2017 によってショーンオバートン 14 コメント

December and January were extremely unkind to me. I took a huge loss on 12 月 9 that coincided with the Fed meeting and another big punch in January. In total, I went from a 28% profit to a ~4% net loss.

Deservedly, my inbox quickly flooded with comments and suggestions on the drawdown. The most common of those was to stop trading during news events.

だから… why am I still trading during news events? There are a few answers to that question.

カーブフィッティング

It’s not like the strategy loses money on every single news event. それは、します。 100% true that news events like the Fed meeting can and badly hurt. Say that I’m determined to exclude news events in the future. I’d have to

Collect historical news event data
Create a second algorithm, which selects the news events that forbid and allow trading to continue
Test how the news algorithm interacts with Dominari
Repeat this many times until I’m happy with the final result

Due to the tiny number of news events that impact the markets like the December 9th announcement, my data set is miniature. The risk of overfitting to historical news events is huge.

Working with tiny amounts of data provides little in the way of long run confidence. Focusing my efforts elsewhere is far more likely to improve performance and requires much less work.

Too many trades

Too many trades sounds a bit naive, so let’s dig into what that means. Dominari trades a portfolio of 7 異なる機器. All instruments cross with USD.

ユーロドル
GBPUSD
USDCHF
AUDUSD
NZDUSD
USDJPY
USDCAD

Many subscribers correctly observed that the major losses occurred with trades open on all 7 pairs in the portfolio at the same time. A good predictor of trade performance is the number of trades open simultaneously.

1-3 trades seems to be consistently profitable
4-5 trades leads to biting my nails
6-7 trades is neutral to disastrous

Testing and confirming the max open trades rule was quick and easy. 5+ trades is very dangerous.

Accordingly, Dominari now exits all open trades if there are 5 or more trades open at any given time.

The next feature of Dominari will be a reversal strategy. Dominari was clearly prone to sudden equity changes if 5+ trades were open at the same time.

Make the losses work for us

An obvious counter strategy is to open trades in the opposite direction whenever Dominari would otherwise open too many trades. Testing the idea is very easy.

Coding a Dominari reversal strategy, しかし, would require a major reprogramming of the expert advisor’s code.

The number of trades per year would be miniscule. I doubt that it would average even 1 trade per month.

The idea is that Dominari can be the normal trading strategy. Whenever Dominari opens too many trades, the strategy then switches into reversal mode and trend trades with a simple trailing stop.

Switching direction should mostly reverse the negative trade skewness back in the positive direction. Almost all of the offending trades open at exactly the same time.

If the biggest losing trades opened at different times, there would be the risk of being too late to the party. All blowout trades opening at the same time means that the strategy can realistically reverse 100% of would-be losses into profits.

Sitting at the top of the docket are changes to Pilum. You can expect to hear about those soon so that I can incorporate Pilum into the Dominari signals. Once that and 2 other internal projects are finished, I’ll be able to dedicate the time required to fully implement the Dominari Reversal System.

Equity stop loss

Dominari uses emergency stop losses on all tickets. That is appropriate 99% of the time for individual trades. Those emergency losses reset once per hour in line with the concept of the TODS.

A little of the problem was bad luck. My stops came within a handful of pips of being triggered. Then they reset even further away, which made a bad problem worse.

When all trades move at the same time, then clearly the strategy could suffer extreme losses.

The first attempted solution after the Fed announcement was to add a portfolio level stop loss. The way that I wrote it also updated once per hour. When a second negative movement came in January, I stopped trying to be clever. It’s a flat, 単純です, stupid stop loss. If I lose more than 4% on all open trades, the entire Dominari portfolio goes flat.

I’m still trading Dominari

I still have my money trading the Dominari system; my confidence in the long term performance hasn’t changed, but it obviously requires safeguards. The max number of trades and the portfolio level stop loss will go a long way to limiting the impact of big moves in the future. AND, I should get the counter-strategy developed relatively soon to turn potential frowns upside down.

最後に, many of you questioned why I’ve been so quiet. The honest answer is that I needed some time to process what happened. It’s easy to feel overwhelmed and discouraged when you get knocked down. I needed some time to process what happened.

I also needed time to double check the changes that I made to the portfolio were actually beneficial. It’s very easy to appease traders when they’re upset by rushing out features before they’re thoughtfully considered.

My money is on the line (私は失いました 2,000 euros between the two moves). What hurt my subscribers hurt me, あまりにも.

One line of code makes all the difference

2 月 9, 2017 によってショーンオバートン 4 コメント

I was really excited about my Pilum strategy two months ago. The research looked great and everything was ready to rock and roll. Demo testing began and then… not much happened.

The Quantilator is (mostly) finished, which finally gave me time to circle back and review what happened with Pilum.

Live demo trading of Pilum. 12月 9, 2016 to Feb 7, 2017

The expected outcome was that I would win 75% 時間の. Trades were infrequent, so I thought maybe I’m just having bad luck. But then my win rate remained stuck around 50%. Simple statistical tests told me this was unlikely to be bad luck.

I used the research time to pour over my research code and to compare it with live trades. What I found was that a single line of code (AHHHHHHHHHHHHHHH!) was incorrectly calculating my entry price, dramatically overstating the profits.

、 flawed code produced this equity curve from a single combination of settings:

When the actual, correct result looks like this with those same settings:

The accurate backtest of Pilum

I’ll be honest… I like the flawed backtest a lot more!

The new, single-setting backtest isn’t as good, but it’s still trade-worthy. There are some characteristics that I dislike and features that I love. Let’s dig into those.

What I dislike

The frequency of trades is very low. Out of 19 months there were a total of 43 取引. 43 trades to comprise a backtest on 40+ instruments is a very small number.

If it weren’t for the statistical pattern backing up the frequency, I would not consider the test. しかし, there are 20,000 bars each on the 44 instruments. あります。 880,000 total bars used to analyze whether my Pilum pattern offers any predictive value.

The most valuable predictions, しかし, are also exceptionally rare. That’s why I’m not able to get the trading frequency higher, which would potentially smooth the returns.

What I love

My previous systems like QB Pro and Dominari traded actively for relatively small wins. Trading costs exercised a massive impact on the overall performance.

The accurate backtest of Pilum

Now look again at the correct equity curve (the image to the right). Do you see the final profit of roughly 0.14? That’s a 14% unleveraged return over a 19 ヶ月の期間.

Allocating 2:1 または 3:1 leverage on this strategy could average annual returns of 15-25%.

Detecting hidden risk

A key measure of risk is skewness. You may not use that term yourself, but it’s something most of you already understand. The biggest complaint about people trading Dominari was that the average winner relative to the average loser was heavily skewed towards the losers.

Dominari wins on most months, but when it lost in December it was devastating. I implemented what I thought was a portfolio stop after the December 9th aftermath. Then I had a smaller, but still very painful, loss in January. The portfolio level stop loss of 3% should prevent future blowouts now that I know what goes wrong.

I still believe in Dominari. しかし, I obviously lost the work of most of the year due to those events.

Knowing that skewness is a good measure of blowout risk (even if you’ve never seen it in a backtest, like happened with Dominari), Pilum looks extremely encouraging.

This is a histogram of profit and loss by days. You should notice a few things.

The tallest bar is to the right of 0. That means that the most frequent outcome is winning.

The biggest winning day is dramatically better than the worst losing day. The worst outcome was a loss of 2%. The best outcome is gains near 10% in a single day (unleveraged!).

This is the statistical profile of an idea that’s much more likely to grab an avalanche of profits than it is to get blown out.

It gets even better

Would you say that the blue and red equity curves are highly or loosely correlated? Look closely.

Writing this blog post made me think carefully about the Pilum strategy. I decided that maybe I should see if all of the profits are coming from different settings at the same time. There’s very little risk of overfitting the data as my strategy only has 1 degree of freedom.

The blue bars are the equity curve of Setting 1.

The red bars are for Setting 2.

Do you think these are tightly or loosely correlated?

If you said loosely correlated, then you are correct. Notice how each equity curve shows large jumps of profit. Did you notice how those profit jumps occur on different days?

The blue setting skyrockets on a single day in November 2016. It leaves the red equity curve choking in its dust.

But then, look what happens as I advance into December. The red curve dramatically catches up to the blue curve and even overtakes it.

The correlation between the 2 strategies is only 57%.

Combine multiple settings into 1 portfolio

This is a much nicer equity curve!

Loose correlations are a GIFT. Combining two bumpy equity curves into a single strategy makes the performance much, much smoother.

The percentages of days that are profitable also increases. Setting 1 is profitable on 58.0% of days. Setting 2 is profitable on 53.5% of days.

しかし… combining them makes Pilum profitable on 68.2% of days. 素晴らしい!

That also provides more data, which puts me in a stronger position to analyze the strategy’s skewness. Look at the frequency histograms below. They’re the same type of histograms that I showed you in the first section of this blog post. As you’ll notice, they look a lot different.

Pilum most probable daily profit and loss

The most probable outcome for any given day is a small winner

The tall green bar is the most probable trading outcome for any given day with filled orders. The average day is a positive return of 0-1%.

The small red bar is the worst trading day of the combined strategy.

The small green bars are the best trading days of the combined strategy.

Look how far to the right the green bars go. The largest winner is more than 3x the biggest loss. と, there are so many more large winners compared to losers.

Giant winners are far more likely than comparable losses.

The Plan

I immediately pushed Pilum into live trading this combination of two strategies. I expect that adding a second degree of freedom and running about 30 different versions of the strategy – all with different settings – will add to the performance and smooth the returns even further.

Dominari hasn’t been working on my FXCM account, which is very difficult to accept because the lacking performance seems to be a buried execution issue. Pilum, しかし, trades very infrequently. It’s unlikely that execution quality will make a dramatic difference in the long term outcomes.

だから, I’m going to convert the FXCM account to trading Pilum exclusively. That will be offered as a strategy on Collective2 within the next few weeks, a company with whom I’ve been working closely. Their users are more investor rather than trading oriented – they’re far more likely to view low trading frequency as a good thing. I suspect that most people here have a different opinion and want to see a lot of market action.

I’ll write an update on Dominari shortly.

The Difference Between Optimization and Curve-Fitting

2 月 3, 2014 によってアンドリュー・セルビー Leave a Comment

Optimization and curve-fitting are two terms that are very common among quantitative traders. They are so common that many traders confuse the terms, or use them as synonyms when they actually have very different meanings.

Michael Harris recently published a guest post on System Trader Success that broke down the meaning of each of these terms and explained how they interact with each other. He also shared a process for determining how likely a strategy was to be exposed to a curve-fitting bias that is based on how its parameters are utilized.

Knowing the difference between optimization and curve-fitting can help you avoid exposing your strategy to backtesting biases.

Optimization vs. Curve-Fitting

Michael began by defining each of the two terms individually. What this shows us is that they have subtle differences with respect to each other. Here is how he explains it:

As already mentioned, curve fitting may involve optimization but the latter is a process with a much broader scope and includes many more possibilities than curve-fitting.

Michael looks at strategy optimization from the viewpoint of finding the best collection of entry and exit signals for a backtesting period. He explains that curve-fitting focuses more on the results than the signals that caused the result.

Is Curve-Fitting Really The Problem?

Another interesting point that Michael brings up is that there is no mathematical proof that optimized systems are more likely to fail because they are curve-fit. He suggests that it is possible for any optimized strategy to fail at any point, and that the strategy failure has nothing to do with what parameters the system uses.

He explains that a different form of bias is far more likely to cause a failure:

それにもかかわらず, optimization that causes selection of entry and exit collections is in general a problematic process because it introduces survivorship bias.

Michael argues that in almost every case where an optimized strategy fails, サバイバーシップ・バイアス is more likely to blame than a curve-fitting bias.

How To Gauge Optimized Trading Strategies

While Michael does not believe that curve-fitting failures are nearly as prevalent as many traders believe, he does discuss how some strategies are more likely to be exposed to curve-fitting than others. In order to gauge how likely an optimized strategy is to be exposed to curve-fitting, Michael divides them into three different classes.

The first class contains strategies where the optimized parameters define both the entry and exit signals. These strategies are the most vulnerable to curve-fitting.

The second class contains strategies where just the entry signals are defined by optimized parameters. These strategies are less likely to be exposed to curve-fitting than those in the first class.

The third class contains strategies where the optimized parameters define only the exit signals. These strategies are the least likely to be exposed to curve-fitting.

転送最適化を歩く

1 月 13, 2014 によってショーンオバートン 6 コメント

歩いていたし、ランダムに雨が降り始めた場合, 明日は傘を持って考慮します。? もちろん、.

人々が行動を観察したとき、私はそのような修辞的な質問をする理由は、, それらはそれに応じて応答します. 彼らは何かが再び起こる可能性があることが予想される場合, 彼らは結果の変化に対応するために彼らの行動を変えます.

あなたは外国為替のロボットを考えるとき, 誰もが永遠に動作する戦略を開発するという夢を持っています. これは、変更する必要はありません. 初期設定は、常に動作. それをオンにし、ビーチに移動.

現実, もちろんです, それよりも複雑です.

ウォークフォワード最適化は継続時間の代わりに、静的な設定のいずれかのセットを探して全体最適化

あなたの戦略は、必然的にゆがんで行くときそれはあなたが何をする必要があるかの期待につながります. それはあなたが現在の市場にも驚くほど動作しない戦略を考え出すことは非常に可能性があります. しかし, 過去の天才は、将来の天才を意味するものではありません. あなたの戦略は、もはや将来的に動作しません可能性が常にあります.

何故ですか? それが今日雨が降れば、それはあなたが明日傘を運ぶかもしれないのと同じ理由です. 人々は一貫した方法で行う市場を観察. より多くの人々は、観測を行うと, 人々はそれに取引を開始します. 市場では、これらの変化に対応, あまりにも多くの人々がそれについて耳いるとして、最終的に機会が完全に洗います.

ウォークフォワードテストは、あなたの戦略が洗い流さしたかどうかを決定するプロセスです. データの1セットでテストすることにより, そして、ブラインドセットにそれをテスト, あなた自身にあなたの戦略が悪いかどうかの指標を与えることができます. ウォークフォワードの目標は、あなたの戦略が良好であることを証明することではありません. それはあなたの戦略が悪いことが知られていないことを証明することです.

ウォークフォワードテストのプロセスは非常に簡単です. あなたは、あなたのテストに使用する情報のセットを識別し、最適化. 実際の例を使用して, 今ではの始まりに 2014. だから、多分あなたはからデータを見て、テストしたいです 2011 を通じて 2012. それはあなたの中のサンプルデータになります, し、サンプルデータのうち、あなたはすべての可能性があります 2013.

歩行フォワードテストを行うために, あなたはあなたの戦略をテストし、分析することになります 2011-2012. その後、, それはだかどうかを判断します “悪いことが知られていません”, あなたは、その後に先に歩きます 2103 見るために性能を確認.

あなたがやったことはブラインドテストであります. あなたはどのように戦略がで実行することになりかわかりませんでした 2013 あなたはそれをテストしたとき 2011-2012. ブラインドサンプルの上に置くことによって、, あなたはそれを失敗する機会を与えます.

それはあなたの最適化の弱点を識別するための絶対的な最高のツールだから非常に多くのトレーダーは徒歩フォワードテストで自分たちの信仰を置く理由は、. あなたは戦略をテストしているとき, それはあなたが過去の機会にオーバーフィットをした可能性が非常に高いです.

自己フィードバックは、現在の市場でループ

私はあなたに例を挙げましょう. 現在の市場では, トレーダーの多くはされています叩い金市場にどこの市場で毎日が開いて開きます。, 彼らはおそらくできる限り金を売却. 場合によっては、数分のスパンでの年間生産量の数倍です. あなたは何を参照してください5または10分間の絶対的自由落下であります. その状態は、一度日間持続します. しかし、それは永遠に続きません. 十分なトレーダーは、人々がオープンに金を強打することを見て起動すると、, 彼らは同じことをやって起動します.

効果的, 誰が彼らのためにその取引を行うために他のトレーダーを教えていた金は、市場のオープンにフォールオフしたいです. 人々は金が開いて最初の5分で落ちることを期待したよう, 彼らはその後、彼らの行動を変えます. いくつかは、オープンを叩いにジャンプし、短い移動しよう.

他の人は自分の動作を変更する開始します. 彼らは、5分間その金無料の落下に注意してください. その後、, 突然止まります, そのようなよりは平均値に戻ります. 非常に多くの分が開いてから経過した彼らは、タックと買いを変え始めましょう. 彼らは販売に先行重いボリュームが最終的に正常に戻ることを期待. 人々は彼らの行動を変更すると, 他の人が親切に対応します.

十分な人々がオープンに販売を開始し、その後、オープン5分で購入する場合, あなたは一人が別のアクションに応答する場合はパターンが形成されていることがわかります. それは日の最初のカップルのために働いていた状態がもはや将来的に機能する自己フィードバックループです.

あなたはこれらの条件に耐えることがある戦略を識別することができた場合, あなたが任意のテストと最適化をしなかった条件に耐えることがあります, あなた自身の将来に成功したのより良いオッズを与えます. それは非常に多くはないトレーダーは、あなたが発見したこの取引チャンスに物事をよく分かっていることを意味し.

フォワードテストを歩くためのアプローチは、として知られている問題に対する解毒剤でありますカーブフィッティング. カーブフィッティングはcould haveの縮約形should haveの縮約形戦略究極のwould haveの縮約形であります. 昨日から、チャートを開いて、私はここで買っただろうと私はここに販売したと言っに似です, すでに蒸散何を知ります.

もちろん、あなたがしようとしています “稼ぐ” 其れでは. あなたは、市場が何をしたか、完璧な情報を知っています. 将来は, あなたは完璧な情報がわかりません. 戦略の目標は、そのあいまいさに対処することです.

新しい状況が必然的に発生していることをあなたが過去の市場の状況にまで完璧にフィットのすべてをしたカーブフィッティング手段, フレーズに似て一種の, “歴史は繰り返さありません, それは韻を踏みます,” あなたの戦略は、同じことを行い.

あなたは過去の実績でもない戦略をしたいです, しかし、あなたは歴史的な市場でお金を稼ぐための戦略を考え出すていません. 戦略を開発する目的は、将来の市場でお金を稼ぐことです. あなたはバックテストしているとき, あなたは、固体の歴史的性能とのバランスをしようとしています, 最も重要なこと, その歴史的知識が将来の業績に外挿することを確認すること. あなたの目標は、お金を稼ぐことです.

ローリングウォークフォワード最適化

ローリングウォークフォワード最適化は徒歩前方にアイデアを取り、継続的に新しいデータにそれを暴露することによって、戦略を改善. それでは、あなたが24ヶ月のサンプル期間を持っているとしましょう. それについて移動するための一つの方法は、二ヶ月の期間、あなたの戦略を最適化することであろう, その後、3ヶ月目に前方に歩いて. あなたは行動を観察し、あなたは、第2、第3の月の再最適化, その後、4番目の月を楽しみにして歩きます.

連続的にそうすることによって, あなたは戦略の減衰時間を排除し、それを継続的な市場の状況に適応する機会を与えます. これは、機械学習に赤毛の継子の一種であります. 経験と損失は戦略に徒歩フォワード最適化により改善し、市場の変化に適応する機会を与えます.

…あなたは戦略の減衰時間を排除し、それを継続的な市場の状況に適応する機会を与えます

散歩フォワード分析のためのもう一つの重要な考慮事項であります自由度系内で. たとえば, 例えば、あなたが、移動averaageクロスを分析しているとしましょう. あなたは、2つの移動平均を使用して、固定stoplossを使用して、利益を取るしています. それはあなたに4度の自由度を与えるだろう. 高速移動平均は、最初の学位であります. ゆっくりと移動平均は、二度あります. 第三はstoplossで、4番目はテイク利益であります.

あなたはシステムで許可するより多くの自由度が大幅に過去のデータにあなたのシステムをフィッティングチャンス0F曲線を増加させ. 絶対的な最高のシステムは、12自由度以下に保ちます. あなたは、大規模な取引の数、あなたが満足すること申し出性能を有する取引の機会を見つけたいです.

あなたの最適化の際に考慮すべきもう一つの要素は、あなたがのために最適化されたものです. ほとんどの人は絶対リターンに焦点を当てます. 戻り値は素晴らしいです, ほとんどのトレーダーは、について多くを気に どうやって 彼らはお金を稼ぐの代わりに、 いくら. 私はあなたに例を挙げましょう. 私が作っシステムを持っていた場合 $25,000 昨年, あなたはそれを望みます? ほとんど誰もがそう言います.

私が作っシステムを使用している場合 $25,000 昨年, しかし、あなたはに負けなければなりませんでした $15,000 あなたはお金を作った前に. ほとんどの人は、そのシステムを望んでいません. これが意味することは、最終的な結果ではなく、日々のパフォーマンスについてより多くを気にしていることです. 最適化、さらには徒歩フォワード最適化の問題点は、必ずしもあなたが現実の世界で気に何に焦点を当てていないということです: あなたがあなたのお金を作っている方法.

ほとんどのチャートパッケージは、ネット結果に焦点を当て、それはあなたのシステム内のいくつかの弱点を引き起こす可能性があります. あなたはレンジ取引している場合, あなたが本当にやったことは桜が少なくとも実質的なニュースの影響を受けている結果を選ぶです. 効果で, あなたはまだ影響を受けていません設定を選択しました脂肪のしっぽ.

あなたはトレンド取引している場合, あなたは正反対をやりました. あなたが意図的に過去に起こった脂肪tailesを最大化の設定を選びます. トレンド取引戦略で, あなたはおそらく安定したパフォーマンスを見つけることするつもりはありません. 代わりに, 何を見つけることは、最適化を頻繁に長い引き起こすことがあります, 絶え間ないドローダウンの継続的な干ばつ. そして、突然、, ほとんどどこからともなく, それはあなたが経験したドローダウンの数倍を返すメガモンスターの勝者を見つけます. これは、架空のbacktestsの罰金です, しかし、現実の世界で、あなたは近く、日常的に損失を被るている場所, ほとんどのトレーダーは、痛みを取ることができません. 私はほとんどの最適化を見つけるの弱点は、彼らはパフォーマンスの一貫性を見ていないということです. 戦略を最適化するための潜在的な代替が見れることになります線形回帰経時株式曲線の. 最高の株式曲線は最強の線形回帰の傾きを有します.

ローリングウォークフォワード最適化を実装する人気チャートパッケージはAmibrokerです, 売買, MultichartsとNinjaTrader.

NinjaTraderでウォークフォワード最適化

コントロールセンターから戦略アナライザを開きます. [ファイル] をクリックします。 / 新機能 / 戦略分析.

NinjaTrader中戦略・アナライザを開きます。

楽器や機器のリストの上でマウスの左クリックと右のマウス、マウスの右クリックメニューを表示します. メニュー項目ウォークフォワードを選択. また、上でクリックすることができます “で” 戦略分析ツールバーのアイコン. あなたがホットキーを好む場合, あなたはまた、Ctrlキーを使用することができます + W. 最後に, あなたもプッシュすることができます “W” 戦略分析の左上のアイコン.
戦略引き出しメニューから戦略を選択します
ウォークフォワードプロパティを設定 (参照してください。 “ウォークフォワードの特性を理解します” プロパティ定義については、以下のセクション) し、[OK]ボタンを押してください.

NinjaTraderに散歩前方最適化を選択するための多くの方法があります

ウォークフォワード進行は、コントロール・センターのステータスバーに表示されます.

どのようにバックテスト偏りディストーションの期待

6 月 10, 2013 によってアンドリュー・セルビー Leave a Comment

“Why don’t my trading results ever seem to match my backtesting results? I followed the system perfectly!”

Anyone who has ever traded a thoroughly backtested systematic strategy has likely asked themselves this question. The answer is that despite all the effort that you put into coding and backtesting your system, there were likely a number of biases that crept in to skew your results.

Backtesting biases come in all shapes and sizes. Some are statistical, while others are psychological. Most stem from incredibly slight discrepancies. They can be very hard to notice, but can distort your expectations. Unchecked biases can lead you to believe that a losing system is relatively safe.

In order to keep these biases out of your backtesting results, it is important to know the most common forms and how they can creep into your testing. Here are the four most common biases that I catch myself struggling with.

Backtesting bias blinds traders to the reality of the markets.

Data Snooping Bias

Data Snooping Bias is also referred to as Optimization Bias or Curve Fitting. This bias is the result of refining too many parameters to improve a system’s performance on a single data set. Like most biases, Data Snooping is fairly easy to understand at face value, however it has a habit of subtly creeping its way into system development.

A common example of data snooping starts out as an honest effort to improve a system. You then test if adding another indicator into the mix will improve your results. If it does, then you incorporate that indicator into the system and test adding another indicator. The end result is a system that is perfectly optimized to trade the exact data set you tested it on. The problem is that the system is only optimized for that specific data set, which already happened.

The best way to avoid data snooping, or curve fitting, is to keep your systems simple, using as few parameters as possible. It is also important to backtest your system on many different data sets across different markets and time periods.

Look-Ahead Bias

Look-Ahead Bias happens when backtesting results are distorted because trading decisions are based on information that was not yet available. This generally happens because extra data is accidentally included and we simply don’t realize it. These errors are often very slight, which is why they are so easily overlooked. They can, しかし, have a dramatic impact on the overall results.

An example of Look-Ahead Bias would be using an entire data set to calculate optimal parameters for a strategy and then testing the strategy on that data set using those parameters. Because the parameters were calculated using the entire data set, they are technically using future data to make trading decisions.

サバイバーシップ・バイアス

Survivorship Bias addresses the concept that many data sets do not include assets that are no longer listed. The result is that these data sets only include the assets that have survived to this point. This makes them skewed to the positive side because they do not include assets that have not survived.

A classic example of Survivorship Bias would be backtesting a stock trading strategy over a time period that included the dot-com crash. If stocks that went bankrupt during that crash are not included in the data set and your strategy would have been trading them, then your results are likely to be skewed.

The best way to avoid this issue is to purchase data that is free of Survivorship Bias. Another option is to use data that is more recent or markets that are less prone to Survivorship Bias.

Tolerance Bias

I’ve seen it written that every trader thinks that they will be able to withstand twice the drawdown that they will actually be able to tolerate. This is the root of Tolerance Bias, which is a trader overestimating his ability to tolerate drawdowns. This bias is completely psychological.

I provided a good example of this in my post about the 3 日高/低平均回帰システム. その記事で, I pointed out that the system had an overall drawdown of just over 15%. The system’s largest losing position was down over 18%.

While it is easy to look at the overall return of the system and agree that it was profitable, it is naive to think that it would not be a challenge to stick to the system at its low point. Imagine sitting on a 15% loss for the life of your system and holding a position that was down 18%. You would certainly be questioning whether markets had fundamentally changed in a way that made your system ineffective.

The best way to avoid tolerance bias is to dramatically underestimate your drawdown tolerance.

Analyzing Pairs with Correlation and Cointegration

5 月 10, 2013 によってルパートにより Leave a Comment

Pairs Trading or Market Neutrality have long been seen as complex hedge fund style strategies with limited application for the retail trader. As part of our series on Correlation and Cointegration, we thought it would be beneficial to look at how both regression patterns can be used effectively to identify pair trading opportunities and scenarios, and how to reduce possible pitfalls.

The mystery that surrounds Cointegration and its complexity from a formulation point of view has somewhat put off many traders. Funnily enough, even as I write this post the spell check does not identify Cointegration as a word, which gives you an indication of how often the term is referenced.

Identifying Good Pairs Trades

Although there are a number of formulas or tests that can be used, one of the most widely adopted is that of the Augmented Dickey Fuller (自動原稿送り装置) Test. Formulating a p-value, the test allows the trader to identify how cointegrated two series are over a specified period in an efficient and simplified way.

To put this into context, if the prices for currency A and currency B are inputted into the model and the p-value comes out at 0.02, then this identifies that 2% of the time the two variables are not stationary or 98% of the time they are cointegrated. Make sure that you use a good number of values (例えば 3 years on a daily chart), otherwise the calculation may not give you the most accurate indication.

There are a number of sites where you can download an excel version of the ADF test including www.quantcode.com

The next part in the calculation process is to work out the 相関関係. It is recommended that multiple time frames are used to also paint a picture of the cycle of the linear regression. To highlight how much of a discrepancy there can be, we ran scans on the EURUSD/GBPUSD and Gold/Silver Pairings. Results were very interesting.

例 1: EURUSD and GBPUSD

30 Day Correlation: 73.46%
2 Year Correlation: 64.89%
13 Year Correlation: 89.20%
2 Year Cointegration: 13%

The long term correlation indicates that the pairs track a very similar path. However on a short term basis, they will gradually move apart. On a cointegration basis, のみ 13% of the time over the last two years have the pairs reverted back to the same mean.

例 2: Gold and Silver

30 Day Correlation: 94.98%
2 Year Correlation: 26.99%
13 Year Correlation: 95.3%
2 Year Cointegration: 85%

The two year interval highlights how the prices are statistically out of their long term and short term ranges. According to the cointegration figure, the prices of both Gold and Silver reverted back to the same mean 85% 時間の.

結論: Gold and Silver is a better pairs trade than EURUSD and GBPUSD.

Cointegration and Correlation Technicals

Now that we determined that gold and silver show the highest correlation and cointegration, we need to analyse the technicals for specific entry and exit points. In the below example, there are three specific graphical representations (top chart is the gold price, middle chart is the regression correlation, and the bottom chart is the Silver price). 、線形回帰 period is 360 days with no look back.

A chart showing gold, silver and their correlation

The diagram had two possible entry and exit points over the one year period. In August, 、 360 regression channel indicated that the linear value would revert to its mean after a period above the 3^rd 標準偏差.

According to the chart, a Short Gold and Long Silver pair signal would have been triggered. The second trade came in December, with the linear line breaking back through the bottom standard deviation channel.

Pitfall

With reference to the above example, we can see a number of issues that could greatly impact performance. カーブフィッティング as it is commonly known is best described as a specific series fitting a time variable without giving a true and clear picture of performance.

Strategies that require definitions from future pricing or large historical data are usually curve fitted. On paper it may look great, but in real trade scenario, the results may be completely different.

、 360 regression channel chart did not pass the out of sample (カーブフィッティング) テスト. As can be seen below, when the look back period is amended to 162 Days, the signal would be completely different.

A moving window for the regression channel provides fewer opportunities for curve fitting

ソリューション

One of the solutions for curve fitting in pairs trading is to reduce the linear regression period to a shorter window or time frame. Although this can result in sensitivity to volatile movements, this reduces the potential risk to forward looking scenarios. Traders should also be aware of changes in the correlation and cointegration values of the pair, as these can shift quickly due to market mispricing or global economic and political events.

曲線の当てはめ

9 月 24, 2012 によってショーンオバートン Leave a Comment

私は一般的に哲学的な理由のために最適化を回避します. トレーダーは、常に一番下の行を気に. 人々は必然的に尋ねる質問です “EAはどのくらい作るん?” 理解できるが、, 私はそのような黒と白の懸念が現実の生活をモデル化しないと信じています.

現実には, 人々はちょうど彼らが作るどのくらいあまりない気. 彼らはまた、彼らはそれを作る方法について多くのことを気に. 違いは、前者を知っているし、後者を無視する傾向があることです. これは、トレーダーが成功したシステムに固執することはできません主な理由です. 彼らは彼らの個人的な疼痛閾値を超えたときはいつでも離れて歩く傾向にあります.

それは1つの重要な要因を無視するため、バランスを最適化することは非常に危険です: それは、特定の期間のための最適な戦略はランダムな結果であることを完全に可能です. ゼロに鼻ダイビングのショート, それはリターンのみに基づいて戦略のメリットを締結することは非常に困難です.

もっと重要なこと, パラメータ最適化のプロセスは、設定の魔法の組み合わせは、それがちょうど発見し取得する物乞いだ存在していることを意味します. これは、使用していることを信じるように平野誤謬です 50 期間移動平均が本当にに比べて決定的な違いを作ります 51 移動平均期間. しかし、私はすべての時間は、歴史的なスイートスポットを探して、すべての可能な結果を排出トレーダーを見ます.

私は数年前に走った一つのテスト, 私はスクリーンショットを保持していたことを望みます, ランダムに純粋に市場に参入し、終了した戦略を書くことでした. 私はの制御を保持する唯一のルールは、周波数でした.

ランダムプロセスは、私の心を吹きました. それは良い方法で最終的でした, が、私は以前に検討し、みなさ戦略と専門委員の多くのことに気づきました “悪くない” 確かに本当に価値がないました. 20 独立した試験では、と巻き上げます 20 異なる結果. 手数料を除きます, 戦略のほとんどはどちらも行わないものお金を失いました. ランダム取引のほとんどは、野生のラウンドトリップを示しました. 利益はアップ実行される可能性があります 10%, に下落 -10%, その後、近くに落ち着きます 0% 戻り値. 収益性の高い戦略が私の心の中で最も際立っていました. そのうちの一つまたは二人は、ほとんどの取引に一貫性のある利益を解約したATM機に似ていました.

ランダムパラメータを使用することのリスクを制限する最も簡単な方法は、意思決定に使用される基礎となる指標を見ることです. 移動平均の上または下価格の交差点, たとえば, 本物を示し、入口と出口の効率を NinjaTrader 評価することができます (70% 範囲バインド出口戦略として使用する場合. エントリはランダムであります). 互いに交差移動平均, しかし, 表示入口と出口の効率の 50%. そのメトリックを使用して, 私は自信を持って窓の外のルールの種類をチャック.

あなたは入り口と出口効率の分析を行い、より大きな何かを見つけた場合は 55%, その時点で最適化は、少なくとも以下の不良追求です. 私は潜在的なパラメータ設定のホットスポットを探すために、過去にそれを使用することに告白します. あなたはその道を行くことにした場合, 提案は聖杯ではなく、評価するように、それらを考慮することが重要です.

Signifigance Testing

5 月 16, 2012 によってショーンオバートン Leave a Comment

I recently had someone from Portugal approach me with an Expert Advisor that he programmed. He felt that it was successful and wanted to get my opinion on its viability. I only know a small amount of information about the system, yet I was able to confidently reject it as unproven.

Statistics are the key. It provides a relatively straightforward toolbox to easily dismiss the simplest of systems. The measure in particular that I care about is called a student’s t-test.

Let’s first talk about when this test is appropriate. If you have fixed take profits and stop losses in place on 100% of all trades, then it is appropriate to use this test. The reason is that these values provide fixed limits on the outcome of trades. The distribution of your trades is likely to form a nice bell curve. In math terms, this is called a Gaussian distribution.

If your strategy uses dynamic exits based on market conditions, then this test is not appropriate. The distribution of forex, stock and futures prices do not follow a bell curve. The test depends on the assumption that the distribution under study follows a bell curve. If you make assumptions that don’t match what you’re measuring, then results are useless at best and dangerously misleading at worst.

If you’d like to go into the mathematics involved, then you can find numerous sources on the internet that explain t-tests. I also really enjoy the book Statistics by Freedman, Pisani and Purves.

Most trading systems do not follow bell curves, which makes the details of the t-test largely irrelevant. What’s useful about it is to show that you can generally feel better about the outcomes and predictions based on the number of results in the バックテスト.

Restrictions and Degrees of Freedoms

The goal of any test is to ensure that the results are accurate. The more frequently that you test a concept, the more confident that you feel about the outcome repeating itself consistently. The idea of predictability largely matches our intuitive expectations. If my co-worker shows up on time regularly, then I feel confident about him showing up on time tomorrow. If he shows up late regularly, then I know that he’s likely to show up late in the future.

The increase in experience increases the level of confidence. 最終的に, the number gets so big that we feel very comfortable with the probabilities.

The Portuguese client presented a system based on moving averages with 3 filters, a stop loss and a take profit. Let’s assume that each filter only had one parameter. The number of restrictions for the buy trade is the moving average period (1), the one parameter for each of the three filters (3), the stop loss distance (1) and the take profit distance (1). This yields a total of 6 restrictions for buy trades. Assuming that sell trades use the same inputs, then we have a total of 12 restrictions.

We can’t begin counting our total number of trades (すなわち, 自由度) until the バックテスト shows at least 12 trades to account for our restrictions. It’s a good idea to not infer anything about a trading system until 30+ trades elapse. と、 12 restrictions in place, that sets the threshold for the minimum number of trades to reach a conclusion at 30 + 12 = 42.

一般的に, it’s a good idea to see 300-400 trades before drawing conclusions about any system. The reason for this is that some events are very rare. They may only occur once every couple of hundred trials. Allowing the amount of information to approach this threshold allows the trader to more comfortably evaluate what hidden risks may be present.

、バックテスト that the Portuguese individual submitted only contained 27 取引. Knowing what we know about basic analysis, I comfortably decided that there is nowhere near enough information on the system to consider evaluating it.

A Word of Caution

An algorithm’s trading statistics are almost certain to change with time. Unless you have a mathematically sophisticated model for evaluating volatility, best practice demands that you evaluate the trading results in light of the type of volatility experienced. Making money in 2008 when the markets nose dived does not mean that you would have made money in 2010, which was much quieter in comparison. Any signal in 2008 that indicated a short would almost certainly show returns that include a handful of monster winners. If those monsters fail to show up because the volatility does not cooperate, then your expert advisor will more than likely flop.

3 Forex System Tips

4 月 23, 2012 によってショーンオバートン Leave a Comment

Many systems nowadays promise profits without effort and easy pips right to your account. You probably know by now that real life doesn’t work that way. A genuinely profitable system is hard to come by. In this article I will describe three simple tips that help you enhance your existing trading systems. The goal is to make them more powerful and accurate.

Stop and Limit Entries to Catch Trends

Many trading systems use market orders to enter and exit the market. Market orders frequently exhibit low entry efficiencies. They get you in the market at a price that is not optimal. Placing a buy or sell order 10 pips from the price you wish to enter, in the direction of the trend for trending systems can make a big difference. For long trades put a buy order 10 pips higher than price, and for short trades put a sell order 10 pips lower than price. Range trading systems might consider limit entries, which would place the orders in the opposite direction of the example above.

Using ATR to Account for Volatility

Many novice system designers use constant pip distances in their forex trading system, すなわち. 15 pips for stop loss, 10 pips for take profit, など. This is a mistake as it doesn’t take into account changes in volatility. If the pair you trade exhibits changes in volatility, the system faces an increased likelihood of failure.

The Average True Range indicator (a.k.a. ATR) gives the average range of a forex pair or stock, and accounts for gaps as well. Instead of using constant numbers, use a percentage of ATR such as 50% ATR or 30% ATR. Once you do this change your system will automatically take into account volatility and will become much more flexible. Such systems will work better and will maintain profitability even in changing market environments.

Avoid Overoptimization

This is a tip especially for the programmers of you: over-optimization is the kiss of death of a trading system. 以上-最適化, a.k.a. カーブフィッティング, means that you add many Forex indicators and filters and use them all to confirm your signals, and optimize them all for maximum profits. In the バックテスト it will seem that your system is becoming better when in fact it will become good only for the past and will fail in any forward-test on real, live data. したがって, it is crucial to only include the most important parts of your system and do not add indicators that don’t make sense at the price-action level. Remember the principle of Occam’s razor: “The simplest solution is usually the most efficient one”.

Michael Wells is an FX programmer and trader. His site contains his insights about Forex trading systems.

専門家アドバイザーを最適化します。

2 月 20, 2012 によってショーンオバートン 1 コメント

One of the lesser known features of the メタトレーダー backtester is the optimization feature. It’s so small that you could be forgiven for overlooking it.

Optimization is the process to maximize a certain outcome. この場合, it’s profit. Any EA developer wants to maximize the amount of profit made over a given period of time. The MetaTrader optimizer allows the trader to search for the combination of inputs that yielded the maximum profit over a given period of time.

The process is identical to running a バックテスト, except that MT4 runs multiple backtests at the same time. It then organizes the results and offers up the best combination.

Telling the backtester to run in optimization mode is easy. Simply put a check next to the word 最適化. MetaTrader will then sort through the combinations that you tell it to consider.

Place a check in the box next to Optimization in the MT4 backtester

The next step is to click on the Expert properties button to the right. A new window appears that contains three tabs: テスト, Inputs and Optimization. These screens allow the trader to inform MetaTrader which variables to consider for testing and how to weight the results.

テスト

The top of the testing section applies to every type of バックテスト. Here you can select the starting balance. MetaTrader defaults the option to $10,000, although you can make this any amount of your choosing.

The second default option allows the trader to restrict the direction of trades. It’s a frequent expert advisor programming request. It’s also one that is unnecessary. Both the backtester and expert advisor options screen allow the trader the option of restricting trades to long only or short only without additional programming. If the EA is not well programmed, this setting may cause errors 4110 または 4100 to appear all over the trading journal. It’s harmless. The only effect should be that the backtester slows down. It’s the result of writing to the journal hundreds of times or more.

The testing tab of the MetaTrader backtester

A groupbox appears underneath these options that inexplicably relates to the optimization process. You’d think it would make more sense to place it in its namesake tab. That’s typical MetaQuotes logic at work.

The first line contains numerous parameters for choosing the best option. User overwhelmingly select for the largest account balance, but other options include the profit factor, expected payoff, maximum drawdown and drawdown percent.

The last line automatically uses a genetic algorithm. Optimization processes use either brute force methods or genetic algorithms. Brute force strikes most people as intuitive although obviously exhausting. The software tests every combination possible. Genetic algorithm’s attempt to make the process more intelligent. When the software sees that certain parameters almost inevitably lead to a losing performance, the algorithm skips similar tests where it expects to lose.

This is a great idea if you have a quality genetic algorithm. My opinion of the メタトレーダー backtester is less than stellar. I don’t feel very confident about the algorithm at all. If you don’t mind spending extra time waiting for test results then I suggest unchecking this option. You don’t want to miss a potentially important combination.

Inputs

Most people find this screen confusing. The first column, called 値, strictly controls inputs for simple backtests. 、値 column is totally ignored during an optimization run.

The inputs tab of the MT4 backtester expert settings

The important columns for this task are Start, ステップ と停止. Start is the lowest number that the MT4 backtester will consider. Step refers to the interval between the lowest value and the highest value. Tightly controlling this setting allows the user to gain quick insights into how changing the variable values affects performance without dragging the tests out for a full week. 停止 is the highest number that the expert advisor will use.

The most obvious candidate for testing in this example is the Take Profit value. The default setting is listed at 50. If you trade the majors, you might want to consider settings ranging between 10 pips and 200 ピップ. That means that you set Take Profit row to 10 ため、 Start column and 200 ため、停止 column. The real trick here is selecting the ステップ. If you choose ステップ = 1, then MetaTrader will run a separate test for every value between 10 と 200. That’s 190 テスト, which is overkill. A step of 10 cuts the total number of tests down to 19.

最適化

This section is the nit-picky part. If a trader feels it’s unacceptable to have 10 consecutive losses in a row, he can place a check next the the Consecutive wins box. MT4 automatically discards any tests which yield a result that contains anything checked off.

The optimization tab in the MT4 backtester expert properties

The optimization tab in the MT4 backtester allows users to discard tests with undesirable traits.

When you finish going through each of the tabs, push OK in the bottom right corner. It’s time to launch the tests.

Curve fitting in the MT4 Optimizer

A word of warning: my personal opinion is that optimizing an expert advisor is usually a very bad idea. The unique settings that yield the most profit in 2012 are unlikely to yield the most profit in 2013. If you don’t control for random chance, there’s a good probability that the 2012 best combination may result in catastrophic losses in 2013.

I recommend that traders pursue any strategy development work in NinjaTrader. I don’t like the idea of optimizing at all. 代わりに, I always focus on testing strategies for 入口と出口の効率. I know from years of experience that these values never fundamentally change on instruments of the charts traded. Entry and exit efficiencies make wonderful metrics for automated trading because they are so stable.