Machine Learning Applied to Market Microstructure - Oxford Man Institute of Quantitative FinanceOxford Man Institute of Quantitative Finance

In brief

At the core of a financial exchange, such as the London Stock Exchange, is its matching engine. Market participants interact with the exchange by placing or cancelling orders to either buy or sell a security of a given quantity at a given price. The exchange collects those orders in what is known as the limit order book. The limit order book describes the supply and demand of a giv-en security at a given moment in time. Orders to sell (buy) at too high (low) prices remain rest-ing in the limit order book until a suitable buy (sell) order arrives at the same or better price, in which case they are matched and a trade is recorded. The part of the exchange which plays those mechanics is called the matching engine - it maintains and updates the state of the limit order book as orders are placed and cancelled, as well as recording trades of matching orders.

In this project we constructed an artificial matching engine which we call the market simulator. The market simulator can replay the market from historical raw market by order data, but it also allows to interact with it by injecting additional orders to the historical market data or by not at all considering historical market data but instead consider interacting agents on an empty exchange where they are the only participants. Being able to perform such studies is essential for understanding price impact. Moreover, the market simulator will also be at the core of any future work on agent based modelling or reinforcement learning at a microstructure level to be done at the OMI. Such works will be instrumental in better understanding liquidity formation in financial markets as well as the different interactions of their participants.

A prototype market simulator was fully implemented in Python. This simulator represents the two sides of the limit order book, the buy and sell side, or bid and ask side, by trees. Trees are used because the number of levels on each side is not fixed and indeed can dynamically grow. At the leaf of each tree a price level of the limit of the order book which

is implemented using a doubly linked list (placements involve adding a new order at one end of the list, matching involves removing a part of one or more orders from the other end and cancellations involve removing an order in the middle of the list – thus the choice of a doubly linked list). To facilitate cancellations a dictionary (hash map) of the price level for each order ID is maintained, allowing to quickly look up where in the tree a given order is located. The Python prototype simulator was mainly designed to reply the existing market data, allowing to convert raw market by order data into L1/L2 data, as well as to produce visualisations and animations of states of the order book through time.

In the second part of the project, the Python prototype simulator was translated to Java, allowing for better performance. At this stage the simulator was set up as a server application which interacts both with the market data as well as with potential agents. The agent’s interaction is done through a client application which connects to the server via sockets. A client can request a state of the market simulator (i.e. a snapshot of the limit order book to a given depth, as well as potential trade records) and take an action based on this (i.e. the placement or cancellation of an order). The client application is written in Java but can also be accessed through a Python API which follows the syntax of the OpenAi Gym API. This provides the foundations for any agent-based or reinforcement learning research at the microstructure level.

More recently, with Peter Belcak, we have been developing a C++ version that benefits from a more monolithic design and efficient implementation. This will allow us to simulate the impact of more realistic factors (such communication delays), various market rules (such as pro-rata markets) and larger agent populations. A code release and accompanying paper are in preparation.