Contents

Preface page vii

Part I Basic probability 1

1 Discrete outcomes 3

1.1 A uniform distribution 3

1.2 Conditional Probabilities. The Bayes Theorem. Independent trials 6

1.3 The exclusion–inclusion formula. The ballot problem 27

1.4 Random variables. Expectation and conditional expectation. Joint distributions 33

1.5 The binomial, Poisson and geometric distributions. Probability generating, moment generating and characteristic functions 54

1.6 Chebyshev’s and Markov’s inequalities. Jensen’s inequality. The Law of Large Numbers and the De Moivre–Laplace Theorem 75

1.7 Branching processes 96

2 Continuous outcomes 108

2.1 Uniform distribution. Probability density functions. Random variables. Independence 108

2.2 Expectation, conditional expectation, variance, generating function, characteristic function 142

2.3 Normal distributions. Convergence of random variables and distributions. The Central Limit Theorem 168

Part II Basic statistics 191

3 Parameter estimation 193

3.1 Preliminaries. Some important probability distributions 193

3.2 Estimators. Unbiasedness 204

3.3 Sufficient statistics. The factorisation criterion 209

3.4 Maximum likelihood estimators 213

3.6 Mean square errors. The Rao–Blackwell Theorem. The Cramér–Rao inequality 218

3.7 Exponential families 225

3.8 Confidence intervals 229

3.9 Bayesian estimation 233

4 Hypothesis testing 242

4.1 Type I and type II error probabilities. Most powerful tests 242

4.2 Likelihood ratio tests. The Neyman–Pearson Lemma and beyond 243

4.3 Goodness of fit. Testing normal distributions, 1: homogeneous samples 252

4.4 The Pearson _2 test. The Pearson Theorem 257

4.5 Generalised likelihood ratio tests. The Wilks Theorem 261

4.6 Contingency tables 270

4.7 Testing normal distributions, 2: non-homogeneous samples 276

4.8 Linear regression. The least squares estimators 289

4.9 Linear regression for normal distributions 292

5 Cambridge University Mathematical Tripos examination questions

in IB Statistics (1992–1999) 298

Appendix 1 Tables of random variables and probability distributions 346

Appendix 2 Index of Cambridge University Mathematical Tripos

examination questions in IA Probability (1992–1999) 349

Bibliography 352