Algorithmic stock trading has become a staple in today's financial market, the majority of trades being now fully automated. Deep Reinforcement Learning (DRL) agents proved to be to a force to be reckon with in many complex games like Chess and Go. We can look at the stock market historical price series and movements as a complex imperfect information environment in which we try to maximize return - profit and minimize risk. This paper reviews the progress made so far with deep reinforcement learning in the subdomain of AI in finance, more precisely, automated low-frequency quantitative stock trading. Many of the reviewed studies had only proof-of-concept ideals with experiments conducted in unrealistic settings and no real-time trading applications. For the majority of the works, despite all showing statistically significant improvements in performance compared to established baseline strategies, no decent profitability level was obtained. Furthermore, there is a lack of experimental testing in real-time, online trading platforms and a lack of meaningful comparisons between agents built on different types of DRL or human traders. We conclude that DRL in stock trading has showed huge applicability potential rivalling professional traders under strong assumptions, but the research is still in the very early stages of development.
The Foreign Exchange (Forex) is a large decentralized market, on which trading analysis and algorithmic trading are popular. Research efforts have been focusing on proof of efficiency of certain technical indicators. We demonstrate, however, that the values of indicator functions are not reproducible and often reduce the number of trade opportunities, compared to price-action trading. In this work, we develop two dataset-agnostic Forex trading heuristic templates with high rate of trading signals. In order to determine most optimal parameters for the given heuristic prototypes, we perform a machine learning simulation of 10 years of Forex price data over three low-margin instruments and 6 different OHLC granularities. As a result, we develop a specific and reproducible list of most optimal trade parameters found for each instrument-granularity pair, with 118 pips of average daily profit for the optimized configuration.
I present an overview of some recent advancements on the empirical analysis and theoretical modeling of the process of price formation in financial markets as the result of the arrival of orders in a limit order book exchange. After discussing critically the possible modeling approaches and the observed stylized facts of order flow, I consider in detail market impact and transaction cost of trades executed incrementally over an extended period of time, by comparing model predictions and recent extensive empirical results. I also discuss how the simultaneous presence of many algorithmic trading executions affects the quality and cost of trading.
This paper develops a machine learning-driven portfolio optimization framework for virtual bidding in electricity markets considering both risk constraint and price sensitivity. The algorithmic trading strategy is developed from the perspective of a proprietary trading firm to maximize profit. A recurrent neural network-based Locational Marginal Price (LMP) spread forecast model is developed by leveraging the inter-hour dependencies of the market clearing algorithm. The LMP spread sensitivity with respect to net virtual bids is modeled as a monotonic function with the proposed constrained gradient boosting tree. We leverage the proposed algorithmic virtual bid trading strategy to evaluate both the profitability of the virtual bid portfolio and the efficiency of U.S. wholesale electricity markets. The comprehensive empirical analysis on PJM, ISO-NE, and CAISO indicates that the proposed virtual bid portfolio optimization strategy considering the price sensitivity explicitly outperforms the one that neglects the price sensitivity. The Sharpe ratio of virtual bid portfolios for all three electricity markets are much higher than that of the S&P 500 index. It was also shown that the efficiency of CAISO's two-settlement system is lower than that of PJM and ISO-NE.
This editorial article partially informs the algorithmic trading community about launching of the new journal "Algorithmic Trading and Controls" (ATC). ATC is an online open-access journal that publishes novel works on algorithmic trading and its control methodologies. In this inaugural article, we discuss nine major challenges that contemporary Algo trading faces. There is nothing superstitiously magical about the number "nine," but so is any other one. Several of these challenges are at the strategy level, including for example, trading of illiquid securities or optimal portfolio execution. Others are more at the level of risk management and controls, such as on how to develop automated controls, testing and simulations. The editorial views could be inevitably personal and biased, but have been explored with the most innocent intention of contributing to this important field in modern financial services and technologies.
We present results demonstrating that an appropriately configured deep learning neural network (DLNN) can automatically learn to be a high-performing algorithmic trading system, operating purely from training-data inputs generated by passive observation of an existing successful trader T. That is, we can point our black-box DLNN system at trader T and successfully have it learn from T's trading activity, such that it trades at least as well as T. Our system, called DeepTrader, takes inputs derived from Level-2 market data, i.e. the market's Limit Order Book (LOB) or Ladder for a tradeable asset. Unusually, DeepTrader makes no explicit prediction of future prices. Instead, we train it purely on input-output pairs where in each pair the input is a snapshot S of Level-2 LOB data taken at the time when T issued a quote Q (i.e. a bid or an ask order) to the market; and DeepTrader's desired output is to produce Q when it is shown S. That is, we train our DLNN by showing it the LOB data S that T saw at the time when T issued quote Q, and in doing so our system comes to behave like T, acting as an algorithmic trader issuing specific quotes in response to specific LOB conditions. We train DeepTrader on large numbers of these S/Q snapshot/quote pairs, and then test it in a variety of market scenarios, evaluating it against other algorithmic trading systems in the public-domain literature, including two that have repeatedly been shown to outperform human traders. Our results demonstrate that DeepTrader learns to match or outperform such existing algorithmic trading systems. We analyse the successful DeepTrader network to identify what features it is relying on, and which features can be ignored. We propose that our methods can in principle create an explainable copy of an arbitrary trader T via "black-box" deep learning methods.
This research analyses high-frequency data of the cryptocurrency market in regards to intraday trading patterns related to algorithmic trading and its impact on the European cryptocurrency market. We study trading quantitatives such as returns, traded volumes, volatility periodicity, and provide summary statistics of return correlations to CRIX (CRyptocurrency IndeX), as well as respective overall high-frequency based market statistics with respect to temporal aspects. Our results provide mandatory insight into a market, where the grand scale employment of automated trading algorithms and the extremely rapid execution of trades might seem to be a standard based on media reports. Our findings on intraday momentum of trading patterns lead to a new quantitative view on approaching the predictability of economic value in this new digital market.
We summarize the fundamental issues at stake in algorithmic trading, and the progress made in this field over the last twenty years. We first present the key problems of algorithmic trading, describing the concepts of optimal execution, optimal placement, and price impact. We then discuss the most recent advances in algorithmic trading through the use of Machine Learning, discussing the use of Deep Learning, Reinforcement Learning, and Generative Adversarial Networks.
The algorithmic trading comes from digitalisation of the processing of trading assets on financial markets. Since 1980 the computerization of the stock market offers real time processing of financial information. This technological revolution has offered processes and mathematic methods to identify best return on transactions. Current research relates to autonomous transaction systems programmed in certain periods and some algorithms. This offers return opportunities where traders can not intervene. There are about thirty algorithms to assist the traders, the best known are the VWAP, the TWAP, TVOL. The algorithms offer the latest strategies and decision-making are the subject of much research. These advances in modeling decision-making autonomous agent can envisage a rich future for these technologies, the players already in use for more than 30% of their trading.
As a fundamental problem in algorithmic trading, order execution aims at fulfilling a specific trading order, either liquidation or acquirement, for a given instrument. Towards effective execution strategy, recent years have witnessed the shift from the analytical view with model-based market assumptions to model-free perspective, i.e., reinforcement learning, due to its nature of sequential decision optimization. However, the noisy and yet imperfect market information that can be leveraged by the policy has made it quite challenging to build up sample efficient reinforcement learning methods to achieve effective order execution. In this paper, we propose a novel universal trading policy optimization framework to bridge the gap between the noisy yet imperfect market states and the optimal action sequences for order execution. Particularly, this framework leverages a policy distillation method that can better guide the learning of the common policy towards practically optimal execution by an oracle teacher with perfect information to approximate the optimal trading strategy. The extensive experiments have shown significant improvements of our method over various strong baselines, with reasonable trading actions.
In the framework of technical analysis for algorithmic trading we use a linear algebra approach in order to define classical technical indicators as bounded operators of the space $l^\infty(\mathbb{N})$. This more abstract view enables us to define in a very simple way the no-lag versions of these tools. Then we apply our results to a basic trading system in order to compare the classical Elder's impulse system with its no-lag version and the so-called Nyquist-Elder's impulse system.
Modern Algorithmic Trading ("Algo") allows institutional investors and traders to liquidate or establish big security positions in a fully automated or low-touch manner. Most existing academic or industrial Algos focus on how to "slice" a big parent order into smaller child orders over a given time horizon. Few models rigorously tackle the actual placement of these child orders. Instead, placement is mostly done with a combination of empirical signals and heuristic decision processes. A self-contained, realistic, and fully functional Child Order Placement (COP) model may never exist due to all the inherent complexities, e.g., fragmentation due to multiple venues, dynamics of limit order books, lit vs. dark liquidity, different trading sessions and rules. In this paper, we propose a reductionism COP model that focuses exclusively on the interplay between placing passive limit orders and sniping using aggressive takeout orders. The dynamic programming model assumes the form of a stochastic linear-quadratic regulator (LQR) and allows closed-form solutions under the backward Bellman equations. Explored in detail are model assumptions and general settings, the choice of state and control variables and the cost functions, and the derivation of the closed-form solutions.
Algorithmic trading is well studied in traditional financial markets. However, it has received less attention in centralized cryptocurrency exchanges. The Commodity Futures Trading Commission (CFTC) attributed the $2010$ flash crash, one of the most turbulent periods in the history of financial markets that saw the Dow Jones Industrial Average lose $9\%$ of its value within minutes, to automated order "spoofing" algorithms. In this paper, we build a set of methodologies to characterize and empirically measure different algorithmic trading strategies in Binance, a large centralized cryptocurrency exchange, using a complete data set of historical trades. We find that a sub-strategy of triangular arbitrage is widespread, where bots convert between two coins through an intermediary coin, and obtain a favorable exchange rate compared to the direct one. We measure the profitability of this strategy, characterize its risks, and outline two strategies that algorithmic trading bots use to mitigate their losses. We find that this strategy yields an exchange ratio that is $0.144\%$, or $14.4$ basis points (bps) better than the direct exchange ratio. $2.71\%$ of all trades on Binance are attributable to this strategy.
Identifying the instances of jumps in a discrete-time-series sample of a jump diffusion model is a challenging task. We have developed a novel statistical technique for jump detection and volatility estimation in a return time series data using a threshold method. The consistency of the volatility estimator has been obtained. Since we have derived the threshold and the volatility estimator simultaneously by solving an implicit equation, we have obtained unprecedented accuracy across a wide range of parameter values. Using this method, the increments attributed to jumps have been removed from a large collection of historical data of Indian sectorial indices. Subsequently, we have tested the presence of regime-switching dynamics in the volatility coefficient using a new discriminating statistic. The statistic has been shown to be sensitive to the transition kernel of the regime-switching model. We perform the testing using the Bootstrap method and find a clear indication of presence of multiple regimes of volatility in the data. A link to all Python codes is given in the conclusion. The methodology is suitable for analyzing high-frequency data and may be applied for algorithmic trading.
Despite the fact that an intraday market price distribution is not normal, the random walk model of price behaviour is as important for the understanding of basic principles of the market as the pendulum model is a starting point of many fundamental theories in physics. This model is a good zero order approximation for liquid fast moving markets where the queue position is less important than the price action. In this paper we present an exact solution for the cost of the static passive slice execution. It is shown, that if a price has a random walk behaviour, there is no optimal limit level for an order execution: all levels have the same execution cost as an immediate aggressive execution at the beginning of the slice. Additionally the estimations for the risk of a limit order as well as the probability of a limit order execution as functions of the slice time and standard deviation of the price are derived.
Working on different aspects of algorithmic trading we empirically discovered a new market invariant. It links together the volatility of the instrument with its traded volume, the average spread and the volume in the order book. The invariant has been tested on different markets and different asset classes. In all cases we did not find significant violation of the invariant. The formula for the invariant was used for the volatility estimation, which we called the instantaneous volatility. Quantitative comparison showed that it reproduces realised volatility better than one-day-ahead GARCH(1,1) prediction. Because of the short-term prediction nature, the instantaneous volatility could be used by algo developers, volatility traders and other market professionals.
Market making is one of the most important aspects of algorithmic trading, and it has been studied quite extensively from a theoretical point of view. The practical implementation of so-called "optimal strategies" however suffers from the failure of most order book models to faithfully reproduce the behaviour of real market participants. This paper is twofold. First, some important statistical properties of order driven markets are identified, advocating against the use of purely Markovian order book models. Then, market making strategies are designed and their performances are compared, based on simulation as well as backtesting. We find that incorporating some simple non-Markovian features in the limit order book greatly improves the performances of market making strategies in a realistic context.
We present the use of the fitted Q iteration in algorithmic trading. We show that the fitted Q iteration helps alleviate the dimension problem that the basic Q-learning algorithm faces in application to trading. Furthermore, we introduce a procedure including model fitting and data simulation to enrich training data as the lack of data is often a problem in realistic application. We experiment our method on both simulated environment that permits arbitrage opportunity and real-world environment by using prices of 450 stocks. In the former environment, the method performs well, implying that our method works in theory. To perform well in the real-world environment, the agents trained might require more training (iteration) and more meaningful variables with predictive value.
Financial markets are often driven by latent factors which traders cannot observe. Here, we address an algorithmic trading problem with collections of heterogeneous agents who aim to perform optimal execution or statistical arbitrage, where all agents filter the latent states of the world, and their trading actions have permanent and temporary price impact. This leads to a large stochastic game with heterogeneous agents. We solve the stochastic game by investigating its mean-field game (MFG) limit, with sub-populations of heterogeneous agents, and, using a convex analysis approach, we show that the solution is characterized by a vector-valued forward-backward stochastic differential equation (FBSDE). We demonstrate that the FBSDE admits a unique solution, obtain it in closed-form, and characterize the optimal behaviour of the agents in the MFG equilibrium. Moreover, we prove the MFG equilibrium provides an $\epsilon$-Nash equilibrium for the finite player game. We conclude by illustrating the behaviour of agents using the optimal MFG strategy through simulated examples.
We propose a microstructural modeling framework for studying optimal market making policies in a FIFO (first in first out) limit order book (LOB). In this context, the limit orders, market orders, and cancel orders arrivals in the LOB are modeled as Cox point processes with intensities that only depend on the state of the LOB. These are high-dimensional models which are realistic from a micro-structure point of view and have been recently developed in the literature. In this context, we consider a market maker who stands ready to buy and sell stock on a regular and continuous basis at a publicly quoted price, and identifies the strategies that maximize her P\&L penalized by her inventory. We apply the theory of Markov Decision Processes and dynamic programming method to characterize analytically the solutions to our optimal market making problem. The second part of the paper deals with the numerical aspect of the high-dimensional trading problem. We use a control randomization method combined with quantization method to compute the optimal strategies. Several computational tests are performed on simulated data to illustrate the efficiency of the computed optimal strategy. In particular, we simulated an order book with constant/ symmet-ric/ asymmetrical/ state dependent intensities, and compared the computed optimal strategy with naive strategies. Some codes are available on https://github.com/comeh.