I was really excited about my Pilum strategy two months ago. The research looked great and everything was ready to rock and roll. Demo testing began and then… not much happened.
The Quantilator is (mostly) finished, which finally gave me time to circle back and review what happened with Pilum.
The expected outcome was that I would win 75% of the time. Trades were infrequent, so I thought maybe I’m just having bad luck. But then my win rate remained stuck around 50%. Simple statistical tests told me this was unlikely to be bad luck.
I used the research time to pour over my research code and to compare it with live trades. What I found was that a single line of code (AHHHHHHHHHHHHHHH!) was incorrectly calculating my entry price, dramatically overstating the profits.
The flawed code produced this equity curve from a single combination of settings:
When the actual, correct result looks like this with those same settings:
I’ll be honest… I like the flawed backtest a lot more!
The new, single-setting backtest isn’t as good, but it’s still trade-worthy. There are some characteristics that I dislike and features that I love. Let’s dig into those.
What I dislike
The frequency of trades is very low. Out of 19 months there were a total of 43 trades. 43 trades to comprise a backtest on 40+ instruments is a very small number.
If it weren’t for the statistical pattern backing up the frequency, I would not consider the test. However, there are 20,000 bars each on the 44 instruments. There are 880,000 total bars used to analyze whether my Pilum pattern offers any predictive value.
The most valuable predictions, however, are also exceptionally rare. That’s why I’m not able to get the trading frequency higher, which would potentially smooth the returns.
What I love
My previous systems like QB Pro and Dominari traded actively for relatively small wins. Trading costs exercised a massive impact on the overall performance.
Now look again at the correct equity curve (the image to the right). Do you see the final profit of roughly 0.14? That’s a 14% unleveraged return over a 19 month period.
Allocating 2:1 or 3:1 leverage on this strategy could average annual returns of 15-25%.
Detecting hidden risk
A key measure of risk is skewness. You may not use that term yourself, but it’s something most of you already understand. The biggest complaint about people trading Dominari was that the average winner relative to the average loser was heavily skewed towards the losers.
Dominari wins on most months, but when it lost in December it was devastating. I implemented what I thought was a portfolio stop after the December 9th aftermath. Then I had a smaller, but still very painful, loss in January. The portfolio level stop loss of 3% should prevent future blowouts now that I know what goes wrong.
I still believe in Dominari. But, I obviously lost the work of most of the year due to those events.
Knowing that skewness is a good measure of blowout risk (even if you’ve never seen it in a backtest, like happened with Dominari), Pilum looks extremely encouraging.
This is a histogram of profit and loss by days. You should notice a few things.
The tallest bar is to the right of 0. That means that the most frequent outcome is winning.
The biggest winning day is dramatically better than the worst losing day. The worst outcome was a loss of 2%. The best outcome is gains near 10% in a single day (unleveraged!).
This is the statistical profile of an idea that’s much more likely to grab an avalanche of profits than it is to get blown out.
It gets even better
Writing this blog post made me think carefully about the Pilum strategy. I decided that maybe I should see if all of the profits are coming from different settings at the same time. There’s very little risk of overfitting the data as my strategy only has 1 degree of freedom.
The blue bars are the equity curve of Setting 1.
The red bars are for Setting 2.
Do you think these are tightly or loosely correlated?
If you said loosely correlated, then you are correct. Notice how each equity curve shows large jumps of profit. Did you notice how those profit jumps occur on different days?
The blue setting skyrockets on a single day in November 2016. It leaves the red equity curve choking in its dust.
But then, look what happens as I advance into December. The red curve dramatically catches up to the blue curve and even overtakes it.
The correlation between the 2 strategies is only 57%.
Combine multiple settings into 1 portfolio
Loose correlations are a GIFT. Combining two bumpy equity curves into a single strategy makes the performance much, much smoother.
The percentages of days that are profitable also increases. Setting 1 is profitable on 58.0% of days. Setting 2 is profitable on 53.5% of days.
But… combining them makes Pilum profitable on 68.2% of days. Awesome!
That also provides more data, which puts me in a stronger position to analyze the strategy’s skewness. Look at the frequency histograms below. They’re the same type of histograms that I showed you in the first section of this blog post. As you’ll notice, they look a lot different.
The tall green bar is the most probable trading outcome for any given day with filled orders. The average day is a positive return of 0-1%.
The small red bar is the worst trading day of the combined strategy.
The small green bars are the best trading days of the combined strategy.
Look how far to the right the green bars go. The largest winner is more than 3x the biggest loss. And, there are so many more large winners compared to losers.
Giant winners are far more likely than comparable losses.
I immediately pushed Pilum into live trading this combination of two strategies. I expect that adding a second degree of freedom and running about 30 different versions of the strategy – all with different settings – will add to the performance and smooth the returns even further.
Dominari hasn’t been working on my FXCM account, which is very difficult to accept because the lacking performance seems to be a buried execution issue. Pilum, however, trades very infrequently. It’s unlikely that execution quality will make a dramatic difference in the long term outcomes.
So, I’m going to convert the FXCM account to trading Pilum exclusively. That will be offered as a strategy on Collective2 within the next few weeks, a company with whom I’ve been working closely. Their users are more investor rather than trading oriented – they’re far more likely to view low trading frequency as a good thing. I suspect that most people here have a different opinion and want to see a lot of market action.
I’ll write an update on Dominari shortly.
Simon R says
Hi Shaun, looks likes you’ve got a very interesting trading system on your hands. I like the analysis output you’ve been getting both quantitatively and qualitatively. I’ve been manually paper charting via Think-or-Swim an “Opening Hour” strategy in Gold and Crude for the last 8 mths. Then plugging the results into Excel, it was quick and easy to do at the time. Have come realise Excel is too impractical to easily manipulate, reformulate and query the data beyond a certain level. Just wondered if you’ve run across anything in the data analytics space that traders would find great to use. Would be good if it had lots of built in analytics, I can customise/code in extras assuming a high level script/language is available. Unfortunately, cost is a factor. Would appreciate your experience in any input you might have. Apologies in advance as I guess this is possibly way off topic. Thanks for all your hard work blogging it’s appreciated.
Shaun Overton says
Thanks for the feedback. The qualitative input was almost as important as the quant angle.
I recommend R, which is what I used for 100% of the initial research for the strategy. You won’t be able to enter spreadsheet-like data cells. However, once those are created, you can get very sophisticated with analyzing the data. You can import the table as a csv file with one line of code like this:
simonsData <- read.csv("myfile.csv", stringsAsFactors=F)
Ramesh Vagadiya says
Great article and appreciate the quant angle that was taken in explaining the results. It’s been about two months since you posted this article, do you have any updates regarding Pilum vs. Dominari and your live trading results?
Shaun Overton says
My plan was thrown off a bit by the forced switch from FXCM to GAIN on Feb 23. It took me until about March 23 to realize that my code had a bug caused by GAIN’s use of suffixes on their currency pair names. So far everything looks ok, but nothing exciting to report. I realistically need another 2-3 months before there’s likely to be anything worth reporting.