1 Review of Probability Theory 18
1.1 Probability spaces and events . . . . . . . . . . . . . . . . . . . . . . 19
1.2 Some elementary properties . . . . . . . . . . . . . . . . . . . . . . 22
1.3 Random variables and expectation values . . . . . . . . . . . . . . . 24
1.4 Properties of the expectation and inequalities . . . . . . . . . . . . . 28
1.5 Limits of random variables . . . . . . . . . . . . . . . . . . . . . . . 31
1.6 Induced measures, independence, and absolute continuity . . . . . . . 37
1.7 A technical tool: Dynkin's -system lemma . . . . . . . . . . . . . . 43
1.8 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2 Conditioning, Martingales, and Stochastic Processes 45
2.1 Conditional expectations and martingales: a trial run . . . . . . . . . 45
2.2 The Radon-Nikodym theorem revisited . . . . . . . . . . . . . . . . 55
2.3 Conditional expectations and martingales for real . . . . . . . . . . . 59
2.4 Some subtleties of continuous time . . . . . . . . . . . . . . . . . . . 65
2.5 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3 The Wiener Process 70
3.1 Basic properties and uniqueness . . . . . . . . . . . . . . . . . . . . 70
3.2 Existence: a multiscale construction . . . . . . . . . . . . . . . . . . 76
3.3 White noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.4 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4 The It.o Integral 87
4.1 What is wrong with the Stieltjes integral? . . . . . . . . . . . . . . . 87
4.2 The It .o integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.3 Some elementary properties . . . . . . . . . . . . . . . . . . . . . . 101
4.4 The It .o calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.5 Girsanov's theorem . . . . . . . . . . . . . . .
4.6 The martingale representation theorem . . . . . . . . . . . . . . . . . 112
4.7 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5 Stochastic Differential Equations 117
5.1 Stochastic differential equations: existence and uniqueness . . . . . . 117
5.2 The Markov property and Kolmogorov's equations . . . . . . . . . . 121
5.3 The Wong-Zakai theorem . . . . . . . . . . . . . . . . . . . . . . . . 125
5.4 The Euler-Maruyama method . . . . . . . . . . . . . . . . . . . . . . 130
5.5 Stochastic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
5.6 Is there life beyond the Lipschitz condition? . . . . . . . . . . . . . . 137
5.7 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6 Optimal Control 141
6.1 Stochastic control problems and dynamic programming . . . . . . . . 141
6.2 Verication: nite time horizon . . . . . . . . . . . . . . . . . . . . . 148
6.3 Verication: indenite time horizon . . . . . . . . . . . . . . . . . . 152
6.4 Verication: innite time horizon . . . . . . . . . . . . . . . . . . . . 156
6.5 The linear regulator . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
6.6 Markov chain approximation . . . . . . . . . . . . . . . . . . . . . . 163
6.7 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7 Filtering Theory 171
7.1 The Bayes formula . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
7.2 Nonlinear ltering for stochastic differential equations . . . . . . . . 177
7.3 The Kalman-Bucy lter . . . . . . . . . . . . . . . . . . . . . . . . . 187
7.4 The Shiryaev-Wonham lter . . . . . . . . . . . . . . . . . . . . . . 194
7.5 The separation principle and LQG control . . . . . . . . . . . . . . . 197
7.6 Transmitting a message over a noisy channel . . . . . . . . . . . . . . 201
7.7 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
8 Optimal Stopping and Impulse Control 207
8.1 Optimal stopping and variational inequalities . . . . . . . . . . . . . 207
8.2 Partial observations: the modication problem . . . . . . . . . . . . . 218
8.3 Changepoint detection . . . . . . . . . . . . . . . . . . . . . . . . . 222
8.4 Hypothesis testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
8.5 Impulse control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
8.6 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240