artificial intelligence Archives - Algorithmic and Mechanical Forex Strategies

How to Use Machine Learning in Your Trading

April 5, 2015 by Shaun Overton 2 Comments

Machine learning presents many unique and compelling advantages for traders looking for an edge in the market. Just in the last year we have seen a huge amount of resources from the world’s top hedge funds, like Bridgewater Associates, dedicated to exploring these techniques.

While using machine learning or artificial intelligence seems incredibly complex and difficult to implement, there are still ways to leverage their capabilities without needing a PhD in math or science.

In this post, we’ll go through 3 different ways that you can use techniques from machine learning to improve your own trading.

Indicator Selection

One of the most important decisions is deciding on which indicators to use to trade. Whether you are a technical or fundamental trader, or you just use price action to trade, your success is going to be largely dependent on the indicators that you use and how you interpret them.

Luckily, there are many different methods for selecting your indicators and this is known as “feature selection” in the machine-learning world.

Using a Decision Tree to Select Your Indicators

Decision trees are very versatile algorithms that have the benefit of being easily interpretable. Given a large dataset of indicators and the price movement of the asset, a decision tree will find the indicators, and indicator values, that best split the data between price increases and price decreases. Indicators closer to the top of the tree are seen as better predictors than those closer to the bottom of the tree, and following a particular branch will allow you to easily find interdependencies and relationships between the indicators.

The decision tree will also give you a set of rules that you can use to trade based on those indicators, but you must be sure to properly prune the tree and test for overfitting.

The decision tree is a powerful, visual tool that can help you decide which combinations of indicators to trade and at what values to trade them. You can find a tutorial on how to build a strategy with a decision tree here or for a more general guide, in R here is a good resource.

Optimization

Once you have the basis for your strategy, the next step is optimization, or choosing the correct parameter values to maximize your chance of success. Many strategies have a wide variety of parameters, such as indicator settings, the entry and exit conditions, stop loss and take profit levels, and position sizing, that make “brute force” methods of trying every single combination extremely difficult and time consuming, if at all even possible.

Solving these types of problems is another area where machine learning excels.

Optimizing a Strategy Using Genetic Algorithms

Genetic algorithms mimic the process of natural selection by creating a unique set of “child” strategies that contains a mixture of the best “parent’” strategies, with a chance of random mutation.

The process begins by encoding your strategy into an array. For example it could read as something like:

Indicator 1 Period

Indicator 2 Period

Buy Condition

Sell Condition

Risk:Reward Ratio

Where a 50 – 200 period moving average cross, with a 50 pip to 100 pip risk-to-reward ratio would be:

50 – period Moving Average

100 – period Moving Average

Indicator 1 crosses above Indicator 2

Indicator 1 crosses below Indicator 2

50:100

You would then generate a large population of strategies with random variations of these parameters. These strategies all have different combinations of moving average periods, entry and exit conditions, and risk-to-reward ratios.

Next, you would test this population by running each strategy over a test set and selecting the top strategies based on a performance metric of your choice.

Finally, you randomly combine the traits of the top strategies, with a small chance of “mutating” some of the parameters, to create a new generation of “child” strategies. You then repeat the evaluation procedure and once again mate the top performing strategies from this new generation. This leads to a survival of fittest scenario where only the top strategies “survive” to pass along their genes to the next generation

Repeat this process a large number of times or until a certain performance criteria is reached and you are left with a very robust strategy built from generations of the best performing strategies!

You do have to make sure that you select an appropriate performance metric (such as risk-adjusted return) and always test the final strategy over data that wasn’t used to build the strategy to ensure that you aren’t overfitting to a particular data set.

This is a very powerful and robust method that has been successful in a wide variety of applications, including the world of trading. You can find a more detailed description here and a tutorial on how to implement it in R here.

Live Trading

One of the more attractive aspects of machine learning is having an algorithm that is able to learn and adapt to changing market conditions. However, this creates a “blackbox” strategy that, if you do not completely understand how the algorithms work and thoroughly tested it yourself, is very difficult to trust on a live account. Not knowing when or why a strategy is entering a trade can be a scary proposition.

However, there are ways to get the benefits of an intelligent, algorithmic approach while still maintaining transparency and understanding in your strategy.

Association Rule Learning

Association Rule Learning is the process of deriving a set of clear, understandable rules from the patterns uncovered by a machine-learning algorithm.

Algorithms, like the Apriori algorithm, search a dataset of indicators, indicator values, and the resulting price movement to produce a set of conditions; basically “if-then” statements, that lead to the highest-performing results. However, it’s still difficult to know exactly where these rules are coming from, the Apriori algorithm requires a fairly large number of parameters to be tuned and this process does not lend itself well to changing market conditions.

With TRAIDE, we took the process one step further and allow you to see the patterns found by an ensemble of machine-learning algorithms, from which you can create your own trading rules. These rules are then easy to implement and adjusted to changing market conditions, all without requiring any programming or mathematical experience. You are able to get the benefits of using machine-learning algorithms to trade while still maintaining complete transparency, an understanding of your strategy, and including your own domain expertise in your trading.

Using machine learning and artificial intelligence to find an edge in the market does not need to be exclusively owned by only the largest financial institutions. As this technology becomes more accessible and these techniques more common, you too can use machine learning to improve your trading.

Tad Slaff

CEO/Co-founder

Inovance

Using Machine Learning to Understand Quantitative Trading

January 20, 2014 by Andrew Selby 2 Comments

Whether or not their strategy has overfit the data it has been tested on should be one of the biggest concerns of any quantitative trader. Overfit strategies will appear to be extremely profitable on the backtesting data, only to fall apart when traded on out-of-sample data. This can lead the the implosion of many young trading accounts.

Andrea from Math Trading wrote an outside-the-box post where he discussed applying techniques from the Machine Learning field in order to explore how the features of a trading strategy related to how overfit the strategy was.

Andrea explains that quantitative trading has many similarities to the Machine Learning field.

In her post, Andrea suggests that trading algorithms are simple a form of artificial intelligence applied to price action. He also explains that the more features that are added to a strategy, the more likely it is to overfit the data it is being tested on.

Building on that idea, he suggests that the features used in a strategy affect different strategies differently. This means that there is a dynamics issue involved as well, so we can’t assume that any system feature will always have the same impact on overfitting. He explains that Machine Learning techniques can be applied to help out with these judgements.

Andrea’s Real World Example

The real world example that Andrea uses to explain this idea is a sporting goods store in Australia. He suggests that because Australian consumers are big fans of watersports, spring and summer would be their best seasons for sales. That basic idea makes sense.

He explains that this seasonal approach would have less predictive power on a sporting good store located in the United States, where there are a number of popular sports in every season of the year. Therefore, the same predictive feature would have dramatically different results based on environment.

Trading System Example

To apply this idea to trading strategies, Andrea uses the rather obvious example of trailing stops. He explains that trailing stops generally work well when they are used in trend following strategies. Those same trailing stops have a much different effect when they are applied to mean reversion strategies.

Because trailing stops work differently in these different situations, we cannot determine that they are either good or bad for overfitting data. They have to be considered with respect to the strategy that they are being used in.

Further Study

After explaining how different features can impact different strategies in different ways, Andrea suggests that there are a few papers in the Machine Learning field that could help to guide our thinking on this topic.

He recommends a paper that gives a basic overview on variable and feature selection. He also recommends Stanford’s Machine Learning Class.

Dark Pools

July 23, 2012 by Shaun Overton Leave a Comment

I picked up Dark Pools by Scott Paterson on Friday evening and finished Sunday afternoon. That ought to say something for the book’s readability. I recommend it to anyone that trades, especially equities traders.

The structure of the stock market is far more complicated than I ever expected. Trading at Interactive Brokers, I always noticed that the execution venue would vary between different acronyms like ISLAND, ARCA and BATS. I knew that they were ECNs, but I never really understood what linked them together.

The stock market is not an exchange in the sense of a centralized location where all transactions occur. It is more like a listing entity where public companies go to list their shares. Actual trading occurs on any network plugged into the system.
The “exchange” is really a complex network of networks with varying degrees of favoritism shown to high volume clients.

Electronic Communication Networks (ECN) originally started in the mid 1980s with the idealistic goal of eliminating the corrupt practices of the NASDAQ floor specialists and market makers. These guys were notorious (and later heavily fined) for colluding to artificially widen spreads on stocks for their own profits. ECNs would cut out the hated middle man while reducing errors, increasing transparency and dramatically decreasing execution time.

As one can imagine, the ECNs took off rather quickly. Not only did they offer much faster execution, but they were also about 60% cheaper to execute a trade.

They corruption that ECNs sought to eliminate inadvertently replaced one problem with another. Networks looking for liquidity offered trading rebates for limit orders that added orders into the system. The setup, which came to be known as maker-taker, created perverse trading incentives for participants. The more trades executed, the more the profits would add up.

Firms sought to become something like Walmart is to wide screen TVs. The more you sell, the more you make. The liquidity providers started fighting aggressively for inside placement of the spread to facilitate ever more trades.

The system spun out of control in two ways. It created a computing arms race where firms focused on purchasing cutting edge technology that shaves microseconds off of calculation time. Firms with the deepest pockets could literally buy an advantage through their computing hardware over the average Joe.

More importantly, the system itself bred its own corruption. The ECNs grew addicted to the liquidity fees. The more trades that fired off, the more money they made. They naturally started catering to their most important clients.

How they did it, though, is what sickens me as a trader. The ECNs started creating order types that were effectively secret. They allowed high speed firms to jump in line over retail chumps using vanilla limit orders. The ECNs offered colocation access at exorbitant fees to give the machines an edge. They allowed the creation of dark liquidity where some players could literally hide orders for execution while others displayed theirs 100% of the time. It makes a complete mockery of the idea of a level playing field.

Artificial intelligence played a big role in how the algorithms operate. What encouraged me, however, was how Patterson chose to wrap up the book. Many of the funds covered near the end start out as relatively small fish working with various types of AI to build predictive trading systems. Their initial results appear encouraging.

One of our biggest projects here is to use various models of fractal markets to build an automated trading system on behalf of the client that I go visit in Ireland so often. Andy and I meet on a near daily basis to discuss our genetic algorithm and how best to encourage the network to behave as we want it to. We are about a month from running our first predictive tests with our in house model. Reading this book encourages me that we are blazing down the right path.