Contents Preface v 1 Mathematical Foundations 1 1.1 Introduction....... 1 1.2 Sets and Set Operations . 2 1.3 Limits of Sequences . . . . 4 1.4 Measurable Spaces, Algebras, and Sets 10 1.5 Measures and Probability Measures . . 18 1.5.1 Measures and Measurable Functions 18 1.6 Integration.................. 26 1.6.1 Miscellaneous Convergence Results 43 1. 7 Extensions to Abstract Spaces 52 1.8 Miscellaneous Concepts . . 64 2 Foundations of Probability 74 2.1 Discrete Models. . . . . . . . . . . . . . . . . 74 2.2 General Probability Models .......... 77 2.2.1 The Measurable Space ( Rn, Bn, Rn) . 77 2.2.2 Specification of Probability Measures . 82 2.2.3 Fubini's Theorem and Miscellaneous Results 92 2.3 Random Variables. . . . . 97 2.3.1 Generalities.... 97 2.3.2 Random Elements 101 2.3.3 Moments of Random Variables and Miscellaneous Inequalities .............. 104 x CONTENTS 2.4 Conditional Probability. . . . . . . . . . . . . . . . . 110 2.4.1 Conditional Probability in Discrete Models. . 110 2.4.2 2.4.3 Conditional Probability in Continuous Models Independence . . . 3 Convergence of Sequences I 3.1 Convergence a.c. and in Probability. 3.1.1 Definitions and Preliminaries 3.1.2 Characterization of Convergence a.c. and Convergence in Probability 3.2 Laws of Large Numbers ..... 3.3 Convergence in Distribution . . . . 3.4 Convergence in Mean of Order p .. 3.5 Relations among Convergence Modes 3.6 Uniform Integrability and Convergence 3.7 Criteria for the SLLN . . . . . . . . . . 3.7.1 3.7.2 Sequences of Independent Random Variables Sequences of Uncorrelated Random Variables 4 Convergence of Sequences II 4.1 Introduction......... 4.2 Properties of Random Elements 4.3 Base and Separability ..... . 4.4 Distributional Aspects of R.E .. 4.4.1 Independence for Random Elements. 4.4.2 Distributions of Random Elements 4.4.3 Moments of Random Elements. 4.4.4 Uncorrelated Random Elements 4.5 Laws of Large Numbers for R.E. . 4.5.1 Preliminaries ....... . 4.5.2 WLLN and SLLN for R.E. . 4.6 Convergence in Probability for R.E. 4.7 Weak Convergence . 4.7.1 Preliminaries ....... . 120 125 133 133 133 . 140 152 154 156 160 166 177 177 191 194 194 196 200 208 208 210 212 216 218 218 219 222 223 223 CONTENTS 4.7.2 Properties of Measures . . . . . . . . 4.7.3 Determining Classes ........ . Xl 226 231 4.7.4 Weak Convergence in Product Space 234 4.8 Convergence in Distribution for R.E. 236 4.8.1 Convergence of Transformed Sequences of R.E. . . . . . . . . . . . . . . . . . . . 239 4.9 Characteristic Functions . 246 4.10 CLT for Independent Random Variables ...................... 257 4.10.1 Preliminaries ..... . 4.10.2 Characteristic Functions for Normal Variables ..... . 4.10.3 Convergence in Probability and Characteristic Functions . . . . 4.10.4 CLT for i.i.d. Random Variables .. 4.10.5 CLT and the Lindeberg Condition. 5 Dependent Sequences 5.1 Preliminaries ........... . 5.2 Definition of Martingale Sequences 5.3 Basic Properties of Martingales 5.4 Square Integrable Sequences 5.5 Stopping Times ..... 5.6 Up crossings . . . . . . . 5.7 Martingale Convergence 5.8 Convergence Sets . . . . 5.9 WLLN and SLLN for Martingales 5.10 Martingale CLT ......... . 5.11 Mixing and Stationary Sequences 5.11.1 Preliminaries and Definitions 5.11.2 Measure Preserving Transformations 5.12 Ergodic Theory ........... . 5.13 Convergence and Ergodicity . . . . . 5.14 Stationary Sequences and Ergodicity . 257 .259 .261 263 265 277 277 279 282 285 289 304 306 310 320 323 338 338 342 344 349 355 XlI CONTENTS 5.14.1 Preliminaries . . . . . . . . . . . . . . . . 355 5.14.2 Convergence and Strict Stationarity. . . . 357 5.14.3 Convergence and Covariance Stationarity. 359 5.15 Miscellaneous Results and Examples ....... 366 Bibliography 371 Index 373