Markov Decision Processes with Applications to Finance

3686

收藏 2011-06-27

Markov Decision Processes with Applications to FinanceSeries: Universitext

Bäuerle, Nicole, Rieder, Ulrich

1st Edition., 2011, XVI, 388 p. 24 illus.

The theory of Markov decision processes focuses on controlled Markov chains in discrete time. The authors establish the theory for general state and action spaces and at the same time show its application by means of numerous examples, mostly taken from the fields of finance and operations research. By using a structural approach many technicalities (concerning measure theory) are avoided. They cover problems with finite and infinite horizons, as well as partially observable Markov decision processes, piecewise deterministic Markov decision processes and stopping problems.
The book presents Markov decision processes in action and includes various state-of-the-art applications with a particular view towards finance. It is useful for upper-level undergraduates, Master's students and researchers in both applied probability and finance, and provides exercises (without solutions).

Content Level » Graduate
Keywords » 90C40, 93E20, 60J05, 91G10, 93E35, 60G40 - Markov Decision Processes - Partially Observable Markov Decision Processes - Portfolio optimization - Stochastic dynamic programming
Related subjects » Applications - Probability Theory and Stochastic Processes - Quantitative Finance

附件列表

Markov Decision Processes with Applications to Finance.pdf

大小:2.88 MB

只需: 5 个论坛币马上下载

扫码加我拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

全部回复

martinnyj

2011-6-27 00:19:37

Contents
1 Introduction and First Examples . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Organization of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Notes and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Part I Finite Horizon Optimization Problems and Financial
Markets
2 Theory of Finite Horizon Markov Decision Processes . . . . . 13
2.1 Markov Decision Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 Finite Horizon Markov Decision Models . . . . . . . . . . . . . . . . . . . 17
2.3 The Bellman Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4 Structured Markov Decision Models . . . . . . . . . . . . . . . . . . . . . . 28
2.4.1 Semicontinuous Markov Decision Models . . . . . . . . . . . . 29
2.4.2 Continuous Markov Decision Models . . . . . . . . . . . . . . . . 32
2.4.3 Measurable Markov Decision Models . . . . . . . . . . . . . . . . 33
2.4.4 Monotone and Convex Markov Decision Models . . . . . . 34
2.4.5 Comparison of Markov Decision Models . . . . . . . . . . . . . 38
2.5 Stationary Markov Decision Models . . . . . . . . . . . . . . . . . . . . . . . 39
2.6 Applications and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.6.1 Red-and-Black Card Game . . . . . . . . . . . . . . . . . . . . . . . . 44
2.6.2 A Cash Balance Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.6.3 Stochastic Linear-Quadratic Problems . . . . . . . . . . . . . . 50
2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.8 Remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3 The Financial Markets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.1 Asset Dynamics and Portfolio Strategies . . . . . . . . . . . . . . . . . . 59
3.2 Jump Markets in Continuous Time . . . . . . . . . . . . . . . . . . . . . . . 66
3.3 Weak Convergence of Financial Markets . . . . . . . . . . . . . . . . . . . 69
3.4 Utility Functions and Expected Utility . . . . . . . . . . . . . . . . . . . . 70
ix
x Contents
3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.6 Remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4 Financial Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.1 The One-Period Optimization Problem. . . . . . . . . . . . . . . . . . . . 76
4.2 Terminal Wealth Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.3 Consumption and Investment Problems . . . . . . . . . . . . . . . . . . . 93
4.4 Optimization Problems with Regime Switching . . . . . . . . . . . . . 100
4.5 Portfolio Selection with Transaction Costs . . . . . . . . . . . . . . . . . 106
4.6 Dynamic Mean-Variance Problems . . . . . . . . . . . . . . . . . . . . . . . . 117
4.7 Dynamic Mean-Risk Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
4.8 Index-Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
4.9 Indifference Pricing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
4.10 Approximation of Continuous-Time Models . . . . . . . . . . . . . . . . 140
4.11 Remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Part II Partially Observable Markov Decision Problems
5 Partially Observable Markov Decision Processes . . . . . . . . . . 147
5.1 Partially Observable Markov Decision Processes . . . . . . . . . . . . 148
5.2 Filter Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
5.3 Reformulation as a Standard Markov Decision Model . . . . . . . 157
5.4 Bayesian Decision Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
5.5 Bandit Problems with Finite Horizon . . . . . . . . . . . . . . . . . . . . . 166
5.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
5.7 Remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
6 Partially Observable Markov Decision Problems in Finance175
6.1 Terminal Wealth Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
6.2 Dynamic Mean-Variance Problems . . . . . . . . . . . . . . . . . . . . . . . . 183
6.3 Remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
Part III Infinite Horizon Optimization Problems
7 Theory of Infinite Horizon Markov Decision Processes . . . . 193
7.1 Markov Decision Models with Infinite Horizon . . . . . . . . . . . . . 194
7.2 Semicontinuous Markov Decision Models . . . . . . . . . . . . . . . . . . 201
7.3 Contracting Markov Decision Models . . . . . . . . . . . . . . . . . . . . . 205
7.4 Positive Markov Decision Models . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.5 Computational Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
7.5.1 Howard’s Policy Improvement Algorithm . . . . . . . . . . . . 212
7.5.2 Linear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
7.5.3 State Space Discretization . . . . . . . . . . . . . . . . . . . . . . . . . 220
7.6 Applications and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
7.6.1 Markov Decision Models with Random Horizon . . . . . . 223
7.6.2 A Cash Balance Problem with Infinite Horizon . . . . . . 224
Contents xi
7.6.3 Casino Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
7.6.4 Bandit Problems with Infinite Horizon . . . . . . . . . . . . . . 230
7.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
7.8 Remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
8 Piecewise Deterministic Markov Decision Processes . . . . . . 243
8.1 Piecewise Deterministic Markov Decision Models . . . . . . . . . . . 243
8.2 Solution via a Discrete-Time Markov Decision Process . . . . . . 247
8.3 Continuous-Time Markov Decision Chains . . . . . . . . . . . . . . . . . 256
8.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
8.5 Remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
9 Optimization Problems in Finance and Insurance . . . . . . . . . 267
9.1 Consumption-Investment Problems with Random Horizon . . . 267
9.2 A Dividend Problem in Risk Theory . . . . . . . . . . . . . . . . . . . . . . 271
9.3 Terminal Wealth Problems in a Pure Jump Market . . . . . . . . . 280
9.4 Trade Execution in Illiquid Markets . . . . . . . . . . . . . . . . . . . . . . . 293
9.5 Remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
Part IV Stopping Problems
10 Theory of Optimal Stopping Problems . . . . . . . . . . . . . . . . . . . . 303
10.1 Stopping Problems with Finite Horizon . . . . . . . . . . . . . . . . . . . 303
10.2 Stopping Problems with Unbounded Horizon . . . . . . . . . . . . . . . 309
10.3 Applications and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
10.3.1 A House Selling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 316
10.3.2 Quiz Show. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
10.3.3 The Secretary Problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
10.3.4 A Bayesian Stopping Problem. . . . . . . . . . . . . . . . . . . . . . 323
10.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
10.5 Remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
11 Stopping Problems in Finance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
11.1 Pricing of American Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
11.2 Credit Granting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
11.3 Remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
Part V Appendix
A Tools from Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
A.1 Semicontinuous Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
A.2 Set-Valued Mappings and a Selection Theorem . . . . . . . . . . . . . 351
A.3 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
xii Contents
B Tools from Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
B.1 Probability Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
B.2 Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
B.3 Stochastic Orders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
C Tools from Mathematical Finance . . . . . . . . . . . . . . . . . . . . . . . . 365
C.1 No Arbitrage Pricing Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
C.2 Risk Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385