【书名】Stochastic Learning and Optimization
【作者】Xi-Ren Cao
【出版社】Springer Science Business Media, LLC.
【版本】1st edition
【出版日期】2007
【文件格式】PDF
【文件大小】2.55M
【页数】574
【ISBN出版号】
【资料类别】控制论
【市面定价】
【扫描版还是影印版】影印版
【是否缺页】否
【关键词】
【内容简介】Learning and optimization of stochastic systems is a multi-disciplinary area
that has attracted wide attention from researchers in many disciplines including
control systems, operations research, and computer science.
【目录】
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Part I Four Disciplines in Learning and Optimization
2 Perturbation Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3 Learning and Optimization with Perturbation Analysis . . . . 147
4 Markov Decision Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
5 Sample-Path-Based Policy Iteration . . . . . . . . . . . . . . . . . . . . . . . 253
6 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
7 Adaptive Control Problems as MDPs . . . . . . . . . . . . . . . . . . . . . . 341
Part II The Event-Based Optimization - A New Approach
8 Event-Based Optimization of Markov Systems . . . . . . . . . . . . . 387
9 Constructing Sensitivity Formulas . . . . . . . . . . . . . . . . . . . . . . . . . 455
A Probability and Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . 491
B Stochastic Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507
C Queueing Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519
Notation and Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563
【整理书评】Compared with other books in the area of learning and optimization, this
book is unique in the following aspects.
1. The book covers various disciplines in learning and optimization, including
PA,MDPs, RL, and I&AC, with a unified framework based on a sensitivity
perspective in the policy space. Many results can be explained with the
two types of fundamental sensitivity formulas in a simple way.
2. We emphasize physical interpretations rather than mathematics. With
the intuitive physical explanations, we propose to construct new sensitivity
formulas with performance potentials as building blocks. The physical
intuition may provide insights that complement to other existing approaches.
3. With the unified framework and the construction approach, we introduce
the recently-developed event-based optimization approach; this approach
opens up a research direction in overcoming/alleviating the curse of dimensionality
issue by utilizing the system’s special features.
4. The performance difference-based approach is applied to all the MDP
problems, including ergodic and multi-chain systems, average and discounted
performance criteria, and even bias optimality and nth-bias optimality.
It is shown that the nth-bias optimal policies eventually lead
to the Blackwell optimal policies. This approach provides a simple, intuitively
clear, and comprehensive presentation of all these problems in
MDPs in a unified way. This presentation of MDPs is unique in existing
books.