5 ways in which machine learning can be used to generate more profitable returns including: portfolio optimization, volatility clustering, price prediction, trading bots and analyzing news sentiment.
About Machine Learning in Finance
Machine learning is busy taking over multiple industries today. Wherever you look you hear about “intelligent” speakers, cameras, traffic directions, self-driving cars, personal assistants and even business and scientific processes. What’s behind this explosion of intelligence is the use of machine learning in finance, which can improve from experience without being explicitly programmed.
If you are used to the traditional ways of doing things, the use of machine learning algorithms in your daily workflow can come as a revelation. Having said that, machine learning in finance and trading is no exception. In this blog post, we look at five ways in which machine learning can help in these areas.
In the following posts in this series, we will look at each topic in more detail.
- Price Prediction
Investors who buy and sell assets such as stocks or commodities are in the business of predicting the future price of the stock. However, as a mathematical problem, predictions are tough. This is because the future performance of an asset may not follow historical price trends, and indeed may change drastically based on internal and external factors as time goes on.
As just one example of an external factor, irrational behavior by other investors can change prices unpredictably. Other external factors such as “black swan” or “white swan” events are also not predictable and can have a drastic effect on the market and individual asset prices.
Is it possible to predict future price movement at all? The answer is: it depends. While attempts at using AI for time series modeling have uniformly shown results which are not promising, there are other techniques which are more useful.
As an example, traders use technical indicators to aid in their trading strategies. These are typically simple regression techniques based on linear support and trend lines or momentum-based averaging across various time periods. Still, many successful traders find them helpful in formulating and executing strategies that lead to success
These indicators are examples of price forecasting via regression. And believe it or not, even simple linear regression such as the indicators mentioned above are trusted machine learning techniques 
While truly accurate price forecasting seems out of reach for the time being, it is possible for machine learning to discover patterns and insights from historical price data and provide several tools which implement more sophisticated regression techniques and perform better than linear prediction.
Examples include time series tools such as ARIMA, random forests and different flavors of neural nets.
- Trading Automatically with Bots
If you view trading as a zero-sum game (especially if you are day trading or trading options or futures), you can see how a technique which has been able to beat human players may be useful in this context. Reinforcement learning can be used to create agent-based strategies to trading which actually work better than time series and deep learning-based approaches.
Generally speaking, reinforcement learning can be viewed as an application of machine learning to the problem of optimal control. In the case of trading, the control parameters would include the time of trade and whether to buy or sell. Training complex models end to end for both prediction and control at the same time results in a high degree of intelligence and adaptive behavior, one which can challenge human-level abilities at times.
The algorithm starts by simulating or performing trades in a paper trading environment in order to learn by practicing and to improve itself over time by viewing positive returns as a reward function. Once trained, these trading bots can follow the reinforcement learning algorithm to do real trades in the market and to generate superior returns.
- Analyzing Market Sentiment Through News
As per the stock prices and the market news or other text based information such as quarterly earnings calls often produce large movements in prices. To reduce the latency in time between the emergence of this information and the ability to act on it, machine learning comes in handy. It helps in translating between the unstructured form of human language to quantitative information, which can be fed into a model which may in turn govern automated strategies to take advantage.
The process of computationally identifying and categorizing whether opinions expressed in a piece of text is positive, negative, or neutral. This is typically done using Natural Language Processing or NLP, a subset of machine learning.
Categories may be expanded into further nuanced ranges which can then be quantitatively estimated using regression (if a continuous range) or classification if discrete levels ranging between -1.0 to 1.0 with 0.0 meaning a neutral sentiment.
Analyzing news in this manner, especially across multiple sources can then allow the savvy trader (or rather, the automated strategies built by them) to profit off the timely recognition of market sentiment.
- Portfolio Optimization
Most asset managers (including yourself if you own more than a single stock in your portfolio) are in the business of maximizing returns on the portfolio while minimizing risk or at least keeping risk within the bounds which might have been set. Typically this is done using Modern Portfolio Theory or MPT along with the ratio of expected return divided by risk which is called the Sharpe ratio. 
There are various tools and methods to optimize portfolios that let you maximize the Sharpe ratio and minimize the volatility within the portfolio. However, using machine learning methods can lead to better results.
For example, assets can be grouped based on profitability using clustering or autoencoders, which are machine learning techniques. Also, traditional methods typically use information from the past and do not try to predict future changes in profitability using market movements. This can be done more effectively with machine learning techniques such as Bayesian learning. Or, we can combine both aspects – prediction and optimization – using deep learning.
- Volatility Clustering
Another important task in finance is volatility modeling for trading. Information about volatility is used in many applications including algorithmic trading and portfolio allocation. Using the right volatility strategies applied to the SP500 can achieve 40x the performance over the index itself.
As volatility is not observed directly, it needs to be inferred from price and tick data. These inferred models can vastly improve with the use of machine learning.
One example of this is given by a paper by Artur Sepp in 2018 where he created 40 different volatility models and applied supervised machine learning to analyze the predictions of the real time series data. Finally, he applied reinforcement learning to dynamically select the best model out of the 40 candidates with respect to predictive ability.
Other approaches to volatility clustering include unsupervised learning (see Figure 2), neural nets and evolutionary algorithms.
Closing
The advent of technology has brought not just higher speeds in trading, often necessitating automation of what were previously human-based financial and trading strategies. It is also showing that to be competitive and hence, more profitable, sophisticated systems which delegate analysis, decision-making and execution based on machine learning are essential.
