Lecture Notes

Chapter 1: Decision Theory and Testing Simple Hypotheses

1.1 Deciding between Two Simple Hypotheses: The Neyman-Pearson Lemma, 8 pages. (PDF) 1.2 Decision Theory, 6 pages. (PDF) 1.3 Bayes Decision Theory, 6 pages. (PDF) 1.4* Realizable Rules, 2 pages. (PDF) 1.5 The Sequential Probability Ratio Test, 5 pages. (PDF) 1.6 Sequential Decision Theory, 2 pages. (PDF) 1.7 Proof of Optimality of the SPRT, 9 pages. (PDF) Chapter 2: Sufficiency and Estimation

2.1 Sufficient Statistics, 8 pages. (PDF) 2.2 Estimation and Convexity, 5 pages. (PDF) 2.3 Minimal Sufficiency and the Lehmann-Scheffé Property, 6 pages. (PDF) 2.4 Lower Bounds on Mean-squared Errors: Information Inequalities, 10 pages. (PDF) 2.5 Exponential Families, 13 pages. (PDF) 2.6 Bayes Estimation, 5 pages. (PDF) 2.7 Stein's Phenomenon and James-Stein Estimators, 5 pages. (PDF) 2.8* Continuity at the Boundary for Exponential Families, 3 pages. (PDF) Chapter 3: Bayes, Maximum Likelihood and M-estimation

3.1 Maximum Likelihood Estimates - In Exponential Families, 4 pages. (PDF) 3.2 Likelihood Equations and Errors-in-variables Regression: Solari's Example, 5 pages. (PDF) 3.3 M-estimators and Their Consistency, 8 pages. (PDF) 3.4 M-estimates and Robust Location Estimates, 8 pages. (PDF) 3.44 Robustness, Breakdown Points, and 1-dimensional Location M-estimates, 6 pages. (PDF) 3.5 Consistency of Approximate M-estimators of psi type, 4 pages. (PDF) 3.6 Asymptotic Normality of M-estimates, 8 pages. (PDF) 3.7 Efficiency of Estimators, 11 pages. (PDF) 3.8 Efficiency of Maximum Likelihood Estimators, 4 pages. (PDF) 3.9 A Likelihood Ratio Test for Nested Composite Hypotheses: Wilks's theorem, 5 pages. (PDF) Chapter 4: Asymptotics of Posterior Probabilities and Model Selection

4.1 Convergence of Posteriors, 5 pages. (PDF) Appendices

Appendix A. Uniqueness of Likelihood Ratios, 2 pages. (PDF) Appendix B. Preservation of Dimension by 1-1 Continuous Functions, 1 page. (PDF) Appendix C. Separability of Stochastic Processes, 1 page. (PDF) Appendix D. Mathematical Foundations of Probability Theory, 2 pages. (PDF) Appendix E. Line-fitting by Distance: Errors-in-variables Regression, 3 pages. (PDF) Appendix F. The Lagrange Multiplier Technique, 2 pages. (PDF)