Machine Learning in Finance: Five Ways to Profit

5 ways in which machine learning can be used to generate more profitable returns including: portfolio optimization, volatility clustering, price prediction, trading bots and analyzing news sentiment.

About Machine Learning in Finance 

Machine learning is busy taking over multiple industries today.  Wherever you look you hear about “intelligent” speakers, cameras, traffic directions, self-driving cars, personal assistants and even business and scientific processes.  What’s behind this explosion of intelligence is the use of machine learning in finance, which can improve from experience without being explicitly programmed.

If you are used to the traditional ways of doing things, the use of machine learning algorithms in your daily workflow can come as a revelation.  Having said that, machine learning in finance and trading is no exception. In this blog post, we look at five ways in which machine learning can help in these areas.

In the following posts in this series, we will look at each topic in more detail.

  1. Price Prediction 

Investors who buy and sell assets such as stocks or commodities are in the business of predicting the future price of the stock. However, as a mathematical problem, predictions are tough. This is because the future performance of an asset may not follow historical price trends, and indeed may change drastically based on internal and external factors as time goes on. 

As just one example of an external factor, irrational behavior by other investors can change prices unpredictably. Other external factors such as “black swan” or “white swan” events are also not predictable and can have a drastic effect on the market and individual asset prices.

Is it possible to predict future price movement at all? The answer is: it depends. While attempts at using AI for time series modeling have uniformly shown results which are not promising, there are other techniques which are more useful.

As an example, traders use technical indicators to aid in their trading strategies. These are typically simple regression techniques based on linear support and trend lines or momentum-based averaging across various time periods. Still, many successful traders find them helpful in formulating and executing strategies that lead to success

These indicators are examples of price forecasting via regression. And believe it or not, even simple linear regression such as the indicators mentioned above are trusted machine learning techniques Image for post

While truly accurate price forecasting seems out of reach for the time being, it is possible for machine learning to discover patterns and insights from historical price data and provide several tools which implement more sophisticated regression techniques and perform better than linear prediction.

Examples include time series tools such as ARIMA, random forests and different flavors of neural nets. 

  1. Trading Automatically with Bots

If you view trading as a zero-sum game (especially if you are day trading or trading options or futures), you can see how a technique which has been able to beat human players may be useful in this context. Reinforcement learning can be used to create agent-based strategies to trading which actually work better than time series and deep learning-based approaches.

Generally speaking, reinforcement learning can be viewed as an application of machine learning to the problem of optimal control. In the case of trading, the control parameters would include the time of trade and whether to buy or sell. Training complex models end to end for both prediction and control at the same time results in a high degree of intelligence and adaptive behavior, one which can challenge human-level abilities at times. 

The algorithm starts by simulating or performing trades in a paper trading environment in order to learn by practicing and to improve itself over time by viewing positive returns as a reward function. Once trained, these trading bots can follow the reinforcement learning algorithm to do real trades in the market and to generate superior returns.

  1. Analyzing Market Sentiment Through News

As per the stock prices and the market news or other text based information such as quarterly earnings calls often produce large movements in prices. To reduce the latency in time between the emergence of this information and the ability to act on it, machine learning comes in handy. It helps in translating between the unstructured form of human language to quantitative information, which can be fed into a model which may in turn govern automated strategies to take advantage.

The process of computationally identifying and categorizing whether opinions expressed in a piece of text is positive, negative, or neutral. This is typically done using Natural Language Processing or NLP, a subset of machine learning.

Categories may be expanded into further nuanced ranges which can then be quantitatively estimated using regression (if a continuous range) or classification if discrete levels ranging between -1.0 to 1.0 with 0.0 meaning a neutral sentiment.

Analyzing news in this manner, especially across multiple sources can then allow the savvy trader (or rather, the automated strategies built by them) to profit off the timely recognition of market sentiment.

  1. Portfolio Optimization

Most asset managers (including yourself if you own more than a single stock in your portfolio) are in the business of maximizing returns on the portfolio while minimizing risk  or at least keeping risk within the bounds which might have been set. Typically this is done using Modern Portfolio Theory or MPT along with the ratio of expected return divided by risk which is called the Sharpe ratio
Figure 3. Efficient Frontier for portfolios of differing weights of Google, Toyota, Coke, and Pepsi stock. Red Star: Maximized Sharpe Ratio, Yellow Star: Minimum Volatility, Blue Ticks and Line: Efficient Frontier

There are various tools and methods to optimize portfolios that let you maximize the Sharpe ratio and minimize the volatility within the portfolio. However, using machine learning methods can lead to better results.

For example, assets can be grouped based on profitability using clustering or autoencoders, which are machine learning techniques.  Also, traditional methods typically use information from the past and do not try to predict future changes in profitability using market movements.  This can be done more effectively with machine learning techniques such as Bayesian learning. Or, we can combine both aspects – prediction and optimization – using deep learning.

  1. Volatility Clustering

Another important task in finance is volatility modeling for trading. Information about volatility is used in many applications including algorithmic trading and portfolio allocation. Using the right volatility strategies applied to the SP500 can achieve 40x the performance over the index itself.
Image for post

As volatility is not observed directly, it needs to be inferred from price and tick data. These inferred models can vastly improve with the use of machine learning.

One example of this is given by a paper by Artur Sepp in 2018 where he created 40 different volatility models and applied supervised machine learning to analyze the predictions of the real time series data. Finally, he applied reinforcement learning to dynamically select the best model out of the 40 candidates with respect to predictive ability.

Other approaches to volatility clustering include unsupervised learning (see Figure 2), neural nets and evolutionary algorithms.

Closing

The advent of technology has brought not just higher speeds in trading, often necessitating automation of what were previously human-based financial and trading strategies. It is also showing that to be competitive and hence, more profitable, sophisticated systems which delegate analysis, decision-making and execution based on machine learning are essential.

  • Experfy Insights

    Top articles, research, podcasts, webinars and more delivered to you monthly.

  • Leave a Comment

    Next Post

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    AI & Machine Learning

    Duration Estimation in an A/B Test

    While running an experiment, waiting for data is often the most challenging period as you are likely to get impatient. All you want during that period is for the A/B test to end as quickly as possible so you can go in a  full-scale execution mode. And, the anxiety adds up when you don’t know

    AI & Machine Learning

    Using Responsible AI to Teach the Golden Rule

    Business leaders have a delicate balancing act when it comes to AI. On one hand, according to O’Reilly, 85% of executives across 25 industries are tasked with either evaluating or deploying AI. On the other hand, risks and unintended consequences continue to grow, from Google search results showing offensively skewed results for “black girls”, to

    AI & Machine Learning

    How Online Education In Data Science Helps Students To Become Work-Ready Graduates

    Data science is a relatively new field, but it has become central in all spheres of human activity. So, why the ongoing buzz around data science?  According to reports from IDC, the global ‘datasphere’ will expand to 175 Zettabytes by 2025. Now, that’s a lot of data. But, to get the best out of the