Machine Learning in Algorithmic Trading

Application by Dutch Proprietary Trading Firms and Possible RisksLast updated at 28-09-2023


Executive SummaryParallel to the success of programs such as Deep Blue and AlphaGo, developmentsin artificial intelligence and machine learning grabbed the attention of industriesworldwide, and therefore the attention of supervisors and policy makers. Since then,think-thanks, academics and supervisors have written extensively about machinelearning, and its implications for the financial markets.However, to the best of our knowledge, few studies have been published about theactual use of artificial intelligence or machine learning in algorithmic trading. Also,few supervisors have shared concerns about risks relevant to conduct supervisors, asopposed to risks for financial markets in a more general sense.This AFM aims to do exactly that with this publication: report about the actual useof machine learning as reported by a subset of Dutch proprietary trading firms, andreport about the possible risks relevant to its supervision. The aim of this publicationis to contribute to the public debate, inform academia and other supervisors. Inaddition, the AFM uses the findings in this study to focus its supervision on the mostrelevant risks to its supervision.Please note that observations in this study are based on surveys sent to – andinterviews held with – a subset of Dutch propriety trading firms. All trading firmsused algorithmic trading. Note that the observations in this study are not necessarilyrepresentative of the use of machine learning in algorithmic trading by othersegments in the financial markets (e.g., brokers executing orders for clients). Also, thefindings represent a snapshot of the state of the market in the year 2022.The AFM observes that (see section 2 and 3):1. Clear terminology is required to account for the many nuances in algorithmictrading2. Machine learning is applied on a large scale in algorithmic trading3. Many machine learning models used in algorithmic trading try to predict the priceof a financial instrument4. Machine learning models look primarily at order book data, no fundamentalinformation, and use 100-1.000 features5. Trading firms heavily rely on supervised learning, not (yet) reinforcement learning6. Trading firms find explainability of models less important than performance7. Trading firms see risks of reinforcement learning based trading algorithms to learnunintentional and negative trading behaviourAlso, the AFM observes several possible risks from the use of machine learning inalgorithmic trading (see section 4). Two risks that are especially relevant to a conductsupervisor such as the AFM are:1. Lack of explainability of machine learning models poses challenges for tradingfirms to comply with organisational requirements regarding algorithmic trading2. Increased risk of market manipulation. Reinforcement learning could increasethe risk of trading algorithms learning unintended, negative trading behaviour,while the implicit use of machine learning could make trading algorithms moresusceptible to falling pretty to manipulation