摘要翻译：
人工智能中的一个核心问题是在部分可观察的环境中，在不确定的情况下规划未来报酬的最大化。在本文中，我们提出并演示了一种新的算法，它可以直接从动作-观察对序列中准确地学习这样一个环境的模型。然后，我们通过在学习的模型中规划并恢复一个在原始环境中接近最优的策略来关闭从观察到行动的循环。具体来说，我们提出了一种有效的统计一致性谱算法来学习预测状态表示(PSR)的参数。我们通过学习一个模拟的高维、基于视觉的移动机器人规划任务模型来演示该算法，然后在学习的PSR中执行近似的基于点的规划。分析结果表明，该算法学习的状态空间能够有效地捕捉环境的基本特征。这种表示允许用少量的参数进行准确的预测，并实现成功和高效的规划。
---
英文标题：
《Closing the Learning-Planning Loop with Predictive State Representations》
---
作者：
Byron Boots, Sajid M. Siddiqi, Geoffrey J. Gordon
---
最新提交年份：
2009
---
分类信息：

一级分类：Computer Science        计算机科学
二级分类：Machine Learning        机器学习
分类描述：Papers on all aspects of machine learning research (supervised, unsupervised, reinforcement learning, bandit problems, and so on) including also robustness, explanation, fairness, and methodology. cs.LG is also an appropriate primary category for applications of machine learning methods.
关于机器学习研究的所有方面的论文（有监督的，无监督的，强化学习，强盗问题，等等），包括健壮性，解释性，公平性和方法论。对于机器学习方法的应用，CS.LG也是一个合适的主要类别。
--
一级分类：Computer Science        计算机科学
二级分类：Artificial Intelligence        人工智能
分类描述：Covers all areas of AI except Vision, Robotics, Machine Learning, Multiagent Systems, and Computation and Language (Natural Language Processing), which have separate subject areas. In particular, includes Expert Systems, Theorem Proving (although this may overlap with Logic in Computer Science), Knowledge Representation, Planning, and Uncertainty in AI. Roughly includes material in ACM Subject Classes I.2.0, I.2.1, I.2.3, I.2.4, I.2.8, and I.2.11.
涵盖了人工智能的所有领域，除了视觉、机器人、机器学习、多智能体系统以及计算和语言（自然语言处理），这些领域有独立的学科领域。特别地，包括专家系统，定理证明（尽管这可能与计算机科学中的逻辑重叠），知识表示，规划，和人工智能中的不确定性。大致包括ACM学科类I.2.0、I.2.1、I.2.3、I.2.4、I.2.8和I.2.11中的材料。
--

---
英文摘要：
  A central problem in artificial intelligence is that of planning to maximize future reward under uncertainty in a partially observable environment. In this paper we propose and demonstrate a novel algorithm which accurately learns a model of such an environment directly from sequences of action-observation pairs. We then close the loop from observations to actions by planning in the learned model and recovering a policy which is near-optimal in the original environment. Specifically, we present an efficient and statistically consistent spectral algorithm for learning the parameters of a Predictive State Representation (PSR). We demonstrate the algorithm by learning a model of a simulated high-dimensional, vision-based mobile robot planning task, and then perform approximate point-based planning in the learned PSR. Analysis of our results shows that the algorithm learns a state space which efficiently captures the essential features of the environment. This representation allows accurate prediction with a small number of parameters, and enables successful and efficient planning.
---
PDF链接：
https://arxiv.org/pdf/0912.2385