One of the easiest biases that can impact our results during backtesting is data snooping, which can also be called look-ahead bias. This occurs when we allow price data that wouldn’t have been available at the time to affect a trading decision.
One of the most complicated aspects of quantitative trading is learning how to manage these types of biases and many quant traders spend a lot of time working to eliminate them.
If you went back in time with all of our current price data, how well would your system perform?
In an interesting post titled The Case For Data Snooping, Pete from Shifting Sands takes a different approach. He suggests that if data snooping is used in a certain way, it can actually be quite helpful in testing a system’s potential. He lists three advantages that can be gained from data snooping:
However, it can be a useful tool. If we give our system perfect forward knowledge:
1) We establish an upper bound for performance.
2) We can get a quick read if something is worth pursuing further, and
3) It can help highlight other coding errors.
The first two are pretty closely related. If our wonderful model is built using the values it is trying to predict, and still performs no better than random guessing, it’s probably not worth the effort trying to salvage it.The flip side is when it performs well, that will be as good as it will ever get.
He goes on to explain how data snooping can uncover coding errors:
Firstly, if our subsequent testing on non-snooped data provides comparable performance, we probably have another look ahead bug lurking somewhere.Secondly, things like having amazing accuracy yet still performing poorly is another sign of a bug lurking somewhere.
He goes on to demonstrate how he uses this approach to test his systems for coding errors.
Pete’s concept of intentionally introducing a bias into our backtesting is very interesting. This kind of outside-the-box thinking makes you wonder what other ways we could adjust our thinking and use known biases to our advantage.
Leave a Reply