英文标题：
《Adversarial Deep Reinforcement Learning in Portfolio Management》
---
作者：
Zhipeng Liang, Hao Chen, Junhao Zhu, Kangkang Jiang, Yanran Li
---
最新提交年份：
2018
---
英文摘要：
  In this paper, we implement three state-of-art continuous reinforcement learning algorithms, Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO) and Policy Gradient (PG)in portfolio management. All of them are widely-used in game playing and robot control. What\'s more, PPO has appealing theoretical propeties which is hopefully potential in portfolio management. We present the performances of them under different settings, including different learning rates, objective functions, feature combinations, in order to provide insights for parameters tuning, features selection and data preparation. We also conduct intensive experiments in China Stock market and show that PG is more desirable in financial market than DDPG and PPO, although both of them are more advanced. What\'s more, we propose a so called Adversarial Training method and show that it can greatly improve the training efficiency and significantly promote average daily return and sharpe ratio in back test. Based on this new modification, our experiments results show that our agent based on Policy Gradient can outperform UCRP.
---
中文摘要：
在本文中，我们实现了三种最先进的连续强化学习算法，即投资组合管理中的深层确定性策略梯度（DDPG）、近端策略优化（PPO）和策略梯度（PG）。它们都被广泛应用于游戏和机器人控制中。此外，PPO具有诱人的理论特性，有望在投资组合管理中发挥潜力。我们展示了它们在不同设置下的性能，包括不同的学习率、目标函数、特征组合，以便为参数调整、特征选择和数据准备提供见解。我们还对中国股市进行了深入的实验，结果表明，尽管二者都比较先进，但在金融市场上，PG比DDPG和PPO更可取。此外，我们还提出了一种所谓的对抗式训练方法，并表明该方法可以大大提高训练效率，显著提高平均日回报率和回测夏普比。基于这种新的修改，我们的实验结果表明，基于策略梯度的代理可以优于UCRP。
---
分类信息：

一级分类：Quantitative Finance        数量金融学
二级分类：Portfolio Management        项目组合管理
分类描述：Security selection and optimization, capital allocation, investment strategies and performance measurement
证券选择与优化、资本配置、投资策略与绩效评价
--
一级分类：Computer Science        计算机科学
二级分类：Machine Learning        机器学习
分类描述：Papers on all aspects of machine learning research (supervised, unsupervised, reinforcement learning, bandit problems, and so on) including also robustness, explanation, fairness, and methodology. cs.LG is also an appropriate primary category for applications of machine learning methods.
关于机器学习研究的所有方面的论文（有监督的，无监督的，强化学习，强盗问题，等等），包括健壮性，解释性，公平性和方法论。对于机器学习方法的应用，CS.LG也是一个合适的主要类别。
--
一级分类：Statistics        统计学
二级分类：Machine Learning        机器学习
分类描述：Covers machine learning papers (supervised, unsupervised, semi-supervised learning, graphical models, reinforcement learning, bandits, high dimensional inference, etc.) with a statistical or theoretical grounding
覆盖机器学习论文（监督，无监督，半监督学习，图形模型，强化学习，强盗，高维推理等）与统计或理论基础
--

---
PDF下载：
-->