skew Archives - Algorithmic and Mechanical Forex Strategies

Does SKEW Predict VIX?

December 22, 2017 by Shaun Overton 4 Comments

SKEW should lead VIX, right? Traders get worried about a crash, which might anticipate volatility in the S&P 500.

SKEW is in green.
VIX is in blue.

If SKEW was a perfect predictor of VIX, then you’d expect the blue line to look like the green line with a small gap in between them.

The theory was that SKEW (in green) would pull up VI (in blue).

A quick scan of the chart shows that’s not the case. There are occasions where green spikes up followed by blue, but it intuitively feels to me like a case of cherry picking. Also, notice the largest blue spike around value 500. If SKEW lagged VIX of the other way around.

Just for the sake of being thorough, I measured the cross correlations of If SKEW and VIX using both my smoothed values and the unprocessed ones.

Here’s the cross correlation of the smoothed values.

The cross correlation of smoothed SKEW and VIX

And here’s the cross correlation of the unsmoothed values.

If anything, the hypothesis is backwards. SKEW 18 days ahead of VIX has a -19% correlation. The correlation should be positive and > 40% to carry any substantial meaning. The weak correlation value and the fact that it’s negative that this idea is better tossed in the bin.

Click here to download the data used in this analysis. You’ll noticed that I first normalized the VIX and SKEW values to allow for easier visual comparisions. Because the data is extremely noisy, I applied a 7 day SMA to make visual comparisons easier.

The data used was from October 16, 2013 to December 21, 2017.

What is SKEW?

SKEW, which is another volatility index run by the CBOE, provides a measure of how worried traders are about tail risks.

Here’s the full description directly from the exchange:

The crash of October 1987 sensitized investors to the potential for stock market crashes and forever changed their view of S&P 500^® returns. Investors now realize that S&P 500 tail risk – the risk of outlier returns two or more standard deviations below the mean – is significantly greater than under a lognormal distribution. The Cboe SKEW Index (“SKEW”) is an index derived from the price of S&P 500 tail risk. Similar to VIX^®, the price of S&P 500 tail risk is calculated from the prices of S&P 500 out-of-the-money options.
SKEW typically ranges from 100 to 150. A SKEW value of 100 means that the perceived distribution of S&P 500 log-returns is normal, and the probability of outlier returns is therefore negligible. As SKEW rises above 100, the left tail of the S&P 500 distribution acquires more weight, and the probabilities of outlier returns become more significant. One can estimate these probabilities from the value of SKEW. Since an increase in perceived tail risk increases the relative demand for low strike puts, increases in SKEW also correspond to an overall steepening of the curve of implied volatilities, familiar to option traders as the “skew”.

What is VIX?

This is also taken directly from the exchange:

The Cboe Volatility Index^® (VIX^® Index) is considered by many to be the world’s premier barometer of equity market volatility. The VIX Index is based on real-time prices of options on the S&P 500^® Index (SPX) and is designed to reflect investors’ consensus view of future (30-day) expected stock market volatility. The VIX Index is often referred to as the market’s “fear gauge”.
The VIX Index is the centerpiece of Cboe Global Markets’ volatility franchise, which includes volatility indexes on broad-based stock indexes, exchange traded funds, individual stocks, commodities and several strategy and performance based indexes, as well as tradable volatility contracts, such as VIX options and futures.
These revolutionary volatility products can offer investors effective ways to help manage risk, leverage volatility and diversify a portfolio.

2 Painful hits

February 13, 2017 by Shaun Overton 14 Comments

December and January were extremely unkind to me. I took a huge loss on December 9 that coincided with the Fed meeting and another big punch in January. In total, I went from a 28% profit to a ~4% net loss.

Deservedly, my inbox quickly flooded with comments and suggestions on the drawdown. The most common of those was to stop trading during news events.

So… why am I still trading during news events? There are a few answers to that question.

Curve fitting

It’s not like the strategy loses money on every single news event. It’s 100% true that news events like the Fed meeting can and badly hurt. Say that I’m determined to exclude news events in the future. I’d have to

Collect historical news event data
Create a second algorithm, which selects the news events that forbid and allow trading to continue
Test how the news algorithm interacts with Dominari
Repeat this many times until I’m happy with the final result

Due to the tiny number of news events that impact the markets like the December 9th announcement, my data set is miniature. The risk of overfitting to historical news events is huge.

Working with tiny amounts of data provides little in the way of long run confidence. Focusing my efforts elsewhere is far more likely to improve performance and requires much less work.

Too many trades

Too many trades sounds a bit naive, so let’s dig into what that means. Dominari trades a portfolio of 7 different instruments. All instruments cross with USD.

EURUSD
GBPUSD
USDCHF
AUDUSD
NZDUSD
USDJPY
USDCAD

Many subscribers correctly observed that the major losses occurred with trades open on all 7 pairs in the portfolio at the same time. A good predictor of trade performance is the number of trades open simultaneously.

1-3 trades seems to be consistently profitable
4-5 trades leads to biting my nails
6-7 trades is neutral to disastrous

Testing and confirming the max open trades rule was quick and easy. 5+ trades is very dangerous.

Accordingly, Dominari now exits all open trades if there are 5 or more trades open at any given time.

The next feature of Dominari will be a reversal strategy. Dominari was clearly prone to sudden equity changes if 5+ trades were open at the same time.

Make the losses work for us

An obvious counter strategy is to open trades in the opposite direction whenever Dominari would otherwise open too many trades. Testing the idea is very easy.

Coding a Dominari reversal strategy, however, would require a major reprogramming of the expert advisor’s code.

The number of trades per year would be miniscule. I doubt that it would average even 1 trade per month.

The idea is that Dominari can be the normal trading strategy. Whenever Dominari opens too many trades, the strategy then switches into reversal mode and trend trades with a simple trailing stop.

Switching direction should mostly reverse the negative trade skewness back in the positive direction. Almost all of the offending trades open at exactly the same time.

If the biggest losing trades opened at different times, there would be the risk of being too late to the party. All blowout trades opening at the same time means that the strategy can realistically reverse 100% of would-be losses into profits.

Sitting at the top of the docket are changes to Pilum. You can expect to hear about those soon so that I can incorporate Pilum into the Dominari signals. Once that and 2 other internal projects are finished, I’ll be able to dedicate the time required to fully implement the Dominari Reversal System.

Equity stop loss

Dominari uses emergency stop losses on all tickets. That is appropriate 99% of the time for individual trades. Those emergency losses reset once per hour in line with the concept of the TODS.

A little of the problem was bad luck. My stops came within a handful of pips of being triggered. Then they reset even further away, which made a bad problem worse.

When all trades move at the same time, then clearly the strategy could suffer extreme losses.

The first attempted solution after the Fed announcement was to add a portfolio level stop loss. The way that I wrote it also updated once per hour. When a second negative movement came in January, I stopped trying to be clever. It’s a flat, simple, stupid stop loss. If I lose more than 4% on all open trades, the entire Dominari portfolio goes flat.

I’m still trading Dominari

I still have my money trading the Dominari system; my confidence in the long term performance hasn’t changed, but it obviously requires safeguards. The max number of trades and the portfolio level stop loss will go a long way to limiting the impact of big moves in the future. AND, I should get the counter-strategy developed relatively soon to turn potential frowns upside down.

Lastly, many of you questioned why I’ve been so quiet. The honest answer is that I needed some time to process what happened. It’s easy to feel overwhelmed and discouraged when you get knocked down. I needed some time to process what happened.

I also needed time to double check the changes that I made to the portfolio were actually beneficial. It’s very easy to appease traders when they’re upset by rushing out features before they’re thoughtfully considered.

My money is on the line (I lost 2,000 euros between the two moves). What hurt my subscribers hurt me, too.

One line of code makes all the difference

February 9, 2017 by Shaun Overton 4 Comments

I was really excited about my Pilum strategy two months ago. The research looked great and everything was ready to rock and roll. Demo testing began and then… not much happened.

The Quantilator is (mostly) finished, which finally gave me time to circle back and review what happened with Pilum.

Live demo trading of Pilum. Dec 9, 2016 to Feb 7, 2017

The expected outcome was that I would win 75% of the time. Trades were infrequent, so I thought maybe I’m just having bad luck. But then my win rate remained stuck around 50%. Simple statistical tests told me this was unlikely to be bad luck.

I used the research time to pour over my research code and to compare it with live trades. What I found was that a single line of code (AHHHHHHHHHHHHHHH!) was incorrectly calculating my entry price, dramatically overstating the profits.

The flawed code produced this equity curve from a single combination of settings:

When the actual, correct result looks like this with those same settings:

The accurate backtest of Pilum

I’ll be honest… I like the flawed backtest a lot more!

The new, single-setting backtest isn’t as good, but it’s still trade-worthy. There are some characteristics that I dislike and features that I love. Let’s dig into those.

What I dislike

The frequency of trades is very low. Out of 19 months there were a total of 43 trades. 43 trades to comprise a backtest on 40+ instruments is a very small number.

If it weren’t for the statistical pattern backing up the frequency, I would not consider the test. However, there are 20,000 bars each on the 44 instruments. There are 880,000 total bars used to analyze whether my Pilum pattern offers any predictive value.

The most valuable predictions, however, are also exceptionally rare. That’s why I’m not able to get the trading frequency higher, which would potentially smooth the returns.

What I love

My previous systems like QB Pro and Dominari traded actively for relatively small wins. Trading costs exercised a massive impact on the overall performance.

The accurate backtest of Pilum

Now look again at the correct equity curve (the image to the right). Do you see the final profit of roughly 0.14? That’s a 14% unleveraged return over a 19 month period.

Allocating 2:1 or 3:1 leverage on this strategy could average annual returns of 15-25%.

Detecting hidden risk

A key measure of risk is skewness. You may not use that term yourself, but it’s something most of you already understand. The biggest complaint about people trading Dominari was that the average winner relative to the average loser was heavily skewed towards the losers.

Dominari wins on most months, but when it lost in December it was devastating. I implemented what I thought was a portfolio stop after the December 9th aftermath. Then I had a smaller, but still very painful, loss in January. The portfolio level stop loss of 3% should prevent future blowouts now that I know what goes wrong.

I still believe in Dominari. But, I obviously lost the work of most of the year due to those events.

Knowing that skewness is a good measure of blowout risk (even if you’ve never seen it in a backtest, like happened with Dominari), Pilum looks extremely encouraging.

This is a histogram of profit and loss by days. You should notice a few things.

The tallest bar is to the right of 0. That means that the most frequent outcome is winning.

The biggest winning day is dramatically better than the worst losing day. The worst outcome was a loss of 2%. The best outcome is gains near 10% in a single day (unleveraged!).

This is the statistical profile of an idea that’s much more likely to grab an avalanche of profits than it is to get blown out.

It gets even better

Would you say that the blue and red equity curves are highly or loosely correlated? Look closely.

Writing this blog post made me think carefully about the Pilum strategy. I decided that maybe I should see if all of the profits are coming from different settings at the same time. There’s very little risk of overfitting the data as my strategy only has 1 degree of freedom.

The blue bars are the equity curve of Setting 1.

The red bars are for Setting 2.

Do you think these are tightly or loosely correlated?

If you said loosely correlated, then you are correct. Notice how each equity curve shows large jumps of profit. Did you notice how those profit jumps occur on different days?

The blue setting skyrockets on a single day in November 2016. It leaves the red equity curve choking in its dust.

But then, look what happens as I advance into December. The red curve dramatically catches up to the blue curve and even overtakes it.

The correlation between the 2 strategies is only 57%.

Combine multiple settings into 1 portfolio

This is a much nicer equity curve!

Loose correlations are a GIFT. Combining two bumpy equity curves into a single strategy makes the performance much, much smoother.

The percentages of days that are profitable also increases. Setting 1 is profitable on 58.0% of days. Setting 2 is profitable on 53.5% of days.

But… combining them makes Pilum profitable on 68.2% of days. Awesome!

That also provides more data, which puts me in a stronger position to analyze the strategy’s skewness. Look at the frequency histograms below. They’re the same type of histograms that I showed you in the first section of this blog post. As you’ll notice, they look a lot different.

Pilum most probable daily profit and loss

The most probable outcome for any given day is a small winner

The tall green bar is the most probable trading outcome for any given day with filled orders. The average day is a positive return of 0-1%.

The small red bar is the worst trading day of the combined strategy.

The small green bars are the best trading days of the combined strategy.

Look how far to the right the green bars go. The largest winner is more than 3x the biggest loss. And, there are so many more large winners compared to losers.

Giant winners are far more likely than comparable losses.

The Plan

I immediately pushed Pilum into live trading this combination of two strategies. I expect that adding a second degree of freedom and running about 30 different versions of the strategy – all with different settings – will add to the performance and smooth the returns even further.

Dominari hasn’t been working on my FXCM account, which is very difficult to accept because the lacking performance seems to be a buried execution issue. Pilum, however, trades very infrequently. It’s unlikely that execution quality will make a dramatic difference in the long term outcomes.

So, I’m going to convert the FXCM account to trading Pilum exclusively. That will be offered as a strategy on Collective2 within the next few weeks, a company with whom I’ve been working closely. Their users are more investor rather than trading oriented – they’re far more likely to view low trading frequency as a good thing. I suspect that most people here have a different opinion and want to see a lot of market action.

I’ll write an update on Dominari shortly.

Big change to Dominari

March 9, 2016 by Shaun Overton 24 Comments

I said it here and here and here. The biggest issue with my Dominari is trading costs. Things aren’t going to really take off until I do one of two things.

Reduce the trading costs
Make more money on each trade

I’ve been working on Dominari since around September or October of last year. After racking my brain for months, I more or less wrote off the idea of improving the trade profitability.

That suddenly changed last week on Friday after the market closed. The best reason to trade my own systems live is that the agony of underperforming forces creativity. The feeling reminds me a lot of Daymond John’s (the guy from Shark Tank) new book the Power of Broke. When life isn’t going your way, it’s the resourceful and creative who are best able to get to the top.

Nobody wants to feel broke or under extreme stress. As much as we hate those feelings, they’re often the strongest drivers of performance. That’s how I feel right now with Dominari. I’m so close to getting there and wasn’t sure how to fix that missing ingredient.

If it weren’t for that stress, I would not have had my simple but very powerful insight last Friday.

And please don’t laugh. The change is so dumb and obvious that you’re going to wonder what’s wrong with me. When you’re in the thick of designing a system, the ugly truth is that sometimes you get lost in the weeds. Or to use another botany metaphor, you only see the trees instead of the forest.

My key insight was to slightly modify the exit strategy to use limit orders, whereas previously I only exited based on the close of the bar. I noticed two repeated behaviors that finally beat me over the head enough that the point finally sank in.

The number of occasions where my trade closed in the optimal location seemed to be significantly outweighed by the amount of money left on the table. The key insight for me was realizing where to optimally place that limit order. And for those of you on my newsletter, it happens to be closely related to the Auto Take Profit that I’ve been talking about all week.

Backtest assumptions and results

My operating mantra when doing backtests is to minimize the number of assumptions. Spreads for retail traders have changed dramatically from 2008 to today. I remember working as a broker at FXCM when our typical spread on GBPCHF was something like 8-9 pips. I now routinely pay something like 2 pips. It’s impossible to model what happened in the middle without haphazardly guessing.

I find it far more convincing to analyze the raw signal, both on historical and recent market data, then to interpret whether trading costs are likely to be favorable in today’s markets. “Raw signal” is the ideal signal, one which assumes perfect execution, no slippage, no rollover, no spreads and no commissions. The natural result is that you’re overstating historical performance, but the benefit is that you have a very clear idea whether the core idea is a system capable of predicting the market with reasonable risks.

The total leverage employed in the portfolio is 7:1. If I have a $50,000 trading account and held a position in every currency pair in the portfolio, then the notional value of those trades would equal $350,000 (50k * 7).

Another key point is that I used a fixed position size of $12,500 per trade. The size of the trade never increases or decreases during the backtest, which allows me to isolate the impact of the raw signal without adding the variable of money management.

Here are my trade metrics with version 1 of Dominari. Click the images to view them in full size.

The first version of Dominari had a profit factor of 1.26.

After here’s the change with Dominari version 2.0.

My new version of Dominari increases the profit factor to 1.59 with a significantly lower drawdown.

My best case scenario was to hope that the profit factor would jump another 10 points or thereabouts, maybe stretching the profit factor to 1.35 or thereabouts. It’s incredibly exciting to see the edge over breakeven more than double (going from a $0.26 edge to a $0.59 cent edge).

What I’m most excited about is the skew in the returns. Most mean reversion systems look for an edge but are overwhelmed with the impact of losing trades. That was the case with version 1.

The largest losers outweighed the largest winners in version 1.

This new version of Dominari is the very first mean reversion strategy that I’ve ever developed where the winning tails (ie, the biggest winners) nearly equal the losing tails (the biggest losers). It’s almost always the opposite with mean reversion strategies. Said another way, the risk profile of the extreme outcomes significantly improved with version 2.

The impact of the biggest winners is nearly identical to the biggest losers with version 2.

And the metric that most traders care about the most, drawdown, is wildly improved. Version 1 showed a drawdown of 5.72%. The new version is a fraction of that at 1.77%.

Out of sample backtest for Dominari version 2

The out of sample performance is nearly identical to the in sample performance, despite significantly different market conditions.

When I walked my test out of sample onto recent data, covering 2013-2015, the performance characteristics of version 2 are nearly identical to the in-sample test. The profit factor was identical at 1.59, and the max drawdown was 2.01% for 2013-2015.

Translating the theoretical into expected performance parameters

Again, those metrics above are in the ideal world of perfect execution and no trading costs. The real world performance will have lower returns and higher drawdowns. The advantage to having live trade data is that I can now make some kind of intelligent estimate of my expected trade accuracy and profit factor. Just how overstated are the idealized returns likely to be?

The process that I went through to calculate the expected profit factor in the real world is a 5 step process. I don’t think it’s going to make any sense if I try to write out the steps in conversational English. Instead, I’ve chosen to share a spreadsheet where you can view the step by step process for how extrapolating live trading data into expected performance with the new strategy. Click here to view the spreadsheet.

The expected profit factor for my live trading is expected to be between 1.29 to 1.39. The expected percent accuracy for live trades should jump from 62.55% to 70.8%.

The traders who will get first crack at the Total Access Apprenticeship are those are subscribed to the free newsletter. If you’re not signed up, make sure to fill in your email address in the orange box at the top right of this page.