英文标题：
《Financial Trading as a Game: A Deep Reinforcement Learning Approach》
---
作者：
Chien Yi Huang
---
最新提交年份：
2018
---
英文摘要：
  An automatic program that generates constant profit from the financial market is lucrative for every market practitioner. Recent advance in deep reinforcement learning provides a framework toward end-to-end training of such trading agent. In this paper, we propose an Markov Decision Process (MDP) model suitable for the financial trading task and solve it with the state-of-the-art deep recurrent Q-network (DRQN) algorithm. We propose several modifications to the existing learning algorithm to make it more suitable under the financial trading setting, namely 1. We employ a substantially small replay memory (only a few hundreds in size) compared to ones used in modern deep reinforcement learning algorithms (often millions in size.) 2. We develop an action augmentation technique to mitigate the need for random exploration by providing extra feedback signals for all actions to the agent. This enables us to use greedy policy over the course of learning and shows strong empirical performance compared to more commonly used epsilon-greedy exploration. However, this technique is specific to financial trading under a few market assumptions. 3. We sample a longer sequence for recurrent neural network training. A side product of this mechanism is that we can now train the agent for every T steps. This greatly reduces training time since the overall computation is down by a factor of T. We combine all of the above into a complete online learning algorithm and validate our approach on the spot foreign exchange market.
---
中文摘要：
一个能从金融市场产生持续利润的自动程序对每个市场从业者来说都是有利可图的。深度强化学习的最新进展为此类交易代理的端到端培训提供了一个框架。本文提出了一种适用于金融交易任务的马尔可夫决策过程（MDP）模型，并用最先进的深度递归Q网络（DRQN）算法进行求解。我们对现有的学习算法进行了一些修改，使其更适合金融交易环境，即1。与现代深度强化学习算法中使用的重播内存（通常为数百万）相比，我们使用的重播内存非常小（只有几百个大小）2、我们开发了一种动作增强技术，通过向代理提供所有动作的额外反馈信号来缓解随机探索的需要。这使我们能够在学习过程中使用贪婪策略，与更常用的epsilon贪婪探索相比，它显示出强大的经验性能。然而，在一些市场假设下，这种技术是特定于金融交易的。3、我们对一个较长的序列进行采样，以进行递归神经网络训练。这种机制的一个副产品是，我们现在可以为每个T步骤训练代理。这大大减少了训练时间，因为总体计算量减少了一倍。我们将以上所有内容结合到一个完整的在线学习算法中，并在即期外汇市场上验证了我们的方法。
---
分类信息：

一级分类：Quantitative Finance        数量金融学
二级分类：Trading and Market Microstructure        交易与市场微观结构
分类描述：Market microstructure, liquidity, exchange and auction design, automated trading, agent-based modeling and market-making
市场微观结构，流动性，交易和拍卖设计，自动化交易，基于代理的建模和做市
--
一级分类：Computer Science        计算机科学
二级分类：Machine Learning        机器学习
分类描述：Papers on all aspects of machine learning research (supervised, unsupervised, reinforcement learning, bandit problems, and so on) including also robustness, explanation, fairness, and methodology. cs.LG is also an appropriate primary category for applications of machine learning methods.
关于机器学习研究的所有方面的论文（有监督的，无监督的，强化学习，强盗问题，等等），包括健壮性，解释性，公平性和方法论。对于机器学习方法的应用，CS.LG也是一个合适的主要类别。
--
一级分类：Statistics        统计学
二级分类：Machine Learning        机器学习
分类描述：Covers machine learning papers (supervised, unsupervised, semi-supervised learning, graphical models, reinforcement learning, bandits, high dimensional inference, etc.) with a statistical or theoretical grounding
覆盖机器学习论文（监督，无监督，半监督学习，图形模型，强化学习，强盗，高维推理等）与统计或理论基础
--

---
PDF下载：
-->