Optimization and curve-fitting are two terms that are very common among quantitative traders. They are so common that many traders confuse the terms, or use them as synonyms when they actually have very different meanings.

Michael Harris recently published a guest post on System Trader Success that broke down the meaning of each of these terms and explained how they interact with each other. He also shared a process for determining how likely a strategy was to be exposed to a curve-fitting bias that is based on how its parameters are utilized.

Knowing the difference between optimization and curve-fitting can help you avoid exposing your strategy to backtesting biases.

Optimization vs. Curve-Fitting

Michael began by defining each of the two terms individually. What this shows us is that they have subtle differences with respect to each other. Here is how he explains it:

As already mentioned, curve fitting may involve optimization but the latter is a process with a much broader scope and includes many more possibilities than curve-fitting.

Michael looks at strategy optimization from the viewpoint of finding the best collection of entry and exit signals for a backtesting period. He explains that curve-fitting focuses more on the results than the signals that caused the result.

Is Curve-Fitting Really The Problem?

Another interesting point that Michael brings up is that there is no mathematical proof that optimized systems are more likely to fail because they are curve-fit. He suggests that it is possible for any optimized strategy to fail at any point, and that the strategy failure has nothing to do with what parameters the system uses.

He explains that a different form of bias is far more likely to cause a failure:

それにもかかわらず, optimization that causes selection of entry and exit collections is in general a problematic process because it introduces survivorship bias.

Michael argues that in almost every case where an optimized strategy fails, サバイバーシップ ・ バイアス is more likely to blame than a curve-fitting bias.

How To Gauge Optimized Trading Strategies

While Michael does not believe that curve-fitting failures are nearly as prevalent as many traders believe, he does discuss how some strategies are more likely to be exposed to curve-fitting than others. In order to gauge how likely an optimized strategy is to be exposed to curve-fitting, Michael divides them into three different classes.

The first class contains strategies where the optimized parameters define both the entry and exit signals. These strategies are the most vulnerable to curve-fitting.

The second class contains strategies where just the entry signals are defined by optimized parameters. These strategies are less likely to be exposed to curve-fitting than those in the first class.

The third class contains strategies where the optimized parameters define only the exit signals. These strategies are the least likely to be exposed to curve-fitting.