INTRODUCTION TO THE MATHEMATICAL AND STATISTICAL FOUNDATIONS OF ECONOMETRICS
HERMAN J. BIERENS
Pennsylvania State University

Contents
Preface page xv
1 Probability and Measure 1
1.1 The Texas Lotto 1
1.1.1 Introduction 1
1.1.2 Binomial Numbers 2
1.1.3 Sample Space 3
1.1.4Algebras and Sigma-Algebras of Events 3
1.1.5 Probability Measure 4
1.2 Quality Control 6
1.2.1 Sampling without Replacement 6
1.2.2 Quality Control in Practice 7
1.2.3 Sampling with Replacement 8
1.2.4Limits of the Hypergeometric and Binomial
Probabilities 8
1.3 Why Do We Need Sigma-Algebras of Events ? 10
1.4Proper ties of Algebras and Sigma-Algebras 11
1.4.1 General Properties 11
1.4.2 Borel Sets 14
1.5 Properties of Probability Measures 15
1.6 The Uniform Probability Measure 16
1.6.1 Introduction 16
1.6.2 Outer Measure 17
1.7 Lebesgue Measure and Lebesgue Integral 19
1.7.1 Lebesgue Measure 19
1.7.2 Lebesgue Integral 19
1.8 Random Variables and Their Distributions 20
1.8.1 Random Variables and Vectors 20
1.8.2 Distribution Functions 23
1.9 Density Functions 25
vii
viii Contents
1.10 Conditional Probability, Bayes’ Rule,
and Independence 27
1.10.1 Conditional Probability 27
1.10.2 Bayes’ Rule 27
1.10.3 Independence 28
1.11 Exercises 30
Appendix 1.A – Common Structure of the Proofs of Theorems
1.6 and 1.10 32
Appendix 1.B – Extension of an Outer Measure to a
Probability Measure 32
2 Borel Measurability, Integration, and Mathematical
Expectations 37
2.1 Introduction 37
2.2 Borel Measurability 38
2.3 Integrals of Borel-Measurable Functions with Respect
to a Probability Measure 42
2.4General Measurability and Integrals of Random
Variables with Respect to Probability Measures 46
2.5 Mathematical Expectation 49
2.6 Some Useful Inequalities Involving Mathematical
Expectations 50
2.6.1 Chebishev’s Inequality 51
2.6.2 Holder’s Inequality 51
2.6.3 Liapounov’s Inequality 52
2.6.4Mink owski’s Inequality 52
2.6.5 Jensen’s Inequality 52
2.7 Expectations of Products of Independent Random
Variables 53
2.8 Moment-Generating Functions and Characteristic
Functions 55
2.8.1 Moment-Generating Functions 55
2.8.2 Characteristic Functions 58
2.9 Exercises 59
Appendix 2.A – Uniqueness of Characteristic Functions 61
3 Conditional Expectations 66
3.1 Introduction 66
3.2 Properties of Conditional Expectations 72
3.3 Conditional Probability Measures and Conditional
Independence 79
3.4Conditioning on Increasing Sigma-Algebras 80
Contents ix
3.5 Conditional Expectations as the Best Forecast Schemes 80
3.6 Exercises 82
Appendix 3.A – Proof of Theorem 3.12 83
4 Distributions and Transformations 86
4.1 Discrete Distributions 86
4.1.1 The Hypergeometric Distribution 86
4.1.2 The Binomial Distribution 87
4.1.3 The Poisson Distribution 88
4.1.4 The Negative Binomial Distribution 88
4.2 Transformations of Discrete Random Variables and
Vectors 89
4.3 Transformations of Absolutely Continuous Random
Variables 90
4.4 Transformations of Absolutely Continuous Random
Vectors 91
4.4.1 The Linear Case 91
4.4.2 The Nonlinear Case 94
4.5 The Normal Distribution 96
4.5.1 The Standard Normal Distribution 96
4.5.2 The General Normal Distribution 97
4.6 Distributions Related to the Standard Normal
Distribution 97
4.6.1 The Chi-Square Distribution 97
4.6.2 The Student’s t Distribution 99
4.6.3 The Standard Cauchy Distribution 100
4.6.4 The F Distribution 100
4.7 The Uniform Distribution and Its Relation to the
Standard Normal Distribution 101
4.8 The Gamma Distribution 102
4.9 Exercises 102
Appendix 4.A – Tedious Derivations 104
Appendix 4.B – Proof of Theorem 4.4 106
5 The Multivariate Normal Distribution and Its Application
to Statistical Inference 110
5.1 Expectation and Variance of Random Vectors 110
5.2 The Multivariate Normal Distribution 111
5.3 Conditional Distributions of Multivariate Normal
Random Variables 115
5.4Independence of Linear and Quadratic Transformations
of Multivariate Normal Random Variables 117
x Contents
5.5 Distributions of Quadratic Forms of Multivariate
Normal Random Variables 118
5.6 Applications to Statistical Inference under Normality 119
5.6.1 Estimation 119
5.6.2 Confidence Intervals 122
5.6.3 Testing Parameter Hypotheses 125
5.7 Applications to Regression Analysis 127
5.7.1 The Linear Regression Model 127
5.7.2 Least-Squares Estimation 127
5.7.3 Hypotheses Testing 131
5.8 Exercises 133
Appendix 5.A – Proof of Theorem 5.8 134
6 Modes of Convergence 137
6.1 Introduction 137
6.2 Convergence in Probability and the Weak Law of Large
Numbers 140
6.3 Almost-Sure Convergence and the Strong Law of Large
Numbers 143
6.4The Uniform Law of Large Numbers and Its
Applications 145
6.4.1 The Uniform Weak Law of Large Numbers 145
6.4.2 Applications of the Uniform Weak Law of
Large Numbers 145
6.4.2.1 Consistency of M-Estimators 145
6.4.2.2 Generalized Slutsky’s Theorem 148
6.4.3 The Uniform Strong Law of Large Numbers
and Its Applications 149
6.5 Convergence in Distribution 149
6.6 Convergence of Characteristic Functions 154
6.7 The Central Limit Theorem 155
6.8 Stochastic Boundedness, Tightness, and the Op and op
Notations 157
6.9 Asymptotic Normality of M-Estimators 159
6.10 Hypotheses Testing 162
6.11 Exercises 163
Appendix 6.A – Proof of the Uniform Weak Law of
Large Numbers 164
Appendix 6.B – Almost-Sure Convergence and Strong Laws of
Large Numbers 167
Appendix 6.C – Convergence of Characteristic Functions and
Distributions 174
Contents xi
7 Dependent Laws of Large Numbers and Central Limit
Theorems 179
7.1 Stationarity and the Wold Decomposition 179
7.2 Weak Laws of Large Numbers for Stationary Processes 183
7.3 Mixing Conditions 186
7.4Unifor m Weak Laws of Large Numbers 187
7.4.1 Random Functions Depending on
Finite-Dimensional Random Vectors 187
7.4.2 Random Functions Depending on
Infinite-Dimensional Random Vectors 187
7.4.3 Consistency of M-Estimators 190
7.5 Dependent Central Limit Theorems 190
7.5.1 Introduction 190
7.5.2 A Generic Central Limit Theorem 191
7.5.3 Martingale Difference Central Limit Theorems 196
7.6 Exercises 198
Appendix 7.A – Hilbert Spaces 199
8 Maximum Likelihood Theory 205
8.1 Introduction 205
8.2 Likelihood Functions 207
8.3 Examples 209
8.3.1 The Uniform Distribution 209
8.3.2 Linear Regression with Normal Errors 209
8.3.3 Probit and Logit Models 211
8.3.4The Tobit Model 212
8.4Asymptotic Properties of ML Estimators 214
8.4.1 Introduction 214
8.4.2 First- and Second-Order Conditions 214
8.4.3 Generic Conditions for Consistency and
Asymptotic Normality 216
8.4.4 Asymptotic Normality in the Time Series Case 219
8.4.5 Asymptotic Efficiency of the ML Estimator 220
8.5 Testing Parameter Restrictions 222
8.5.1 The Pseudo t-Test and the Wald Test 222
8.5.2 The Likelihood Ratio Test 223
8.5.3 The Lagrange Multiplier Test 225
8.5.4Selecting a Test 226
8.6 Exercises 226
I Review of Linear Algebra 229
I.1 Vectors in a Euclidean Space 229
I.2 Vector Spaces 232
xii Contents
I.3 Matrices 235
I.4The Inverse and Transpose of a Matrix 238
I.5 Elementary Matrices and Permutation Matrices 241
I.6 Gaussian Elimination of a Square Matrix and the
Gauss–Jordan Iteration for Inverting a Matrix 244
I.6.1 Gaussian Elimination of a Square Matrix 244
I.6.2 The Gauss–Jordan Iteration for Inverting a
Matrix 248
I.7 Gaussian Elimination of a Nonsquare Matrix 252
I.8 Subspaces Spanned by the Columns and Rows
of a Matrix 253
I.9 Projections, Projection Matrices, and Idempotent
Matrices 256
I.10 Inner Product, Orthogonal Bases, and Orthogonal
Matrices 257
I.11 Determinants: Geometric Interpretation and
Basic Properties 260
I.12 Determinants of Block-Triangular Matrices 268
I.13 Determinants and Cofactors 269
I.14In verse of a Matrix in Terms of Cofactors 272
I.15 Eigenvalues and Eigenvectors 273
I.15.1 Eigenvalues 273
I.15.2 Eigenvectors 274
I.15.3 Eigenvalues and Eigenvectors of Symmetric
Matrices 275
I.16 Positive Definite and Semidefinite Matrices 277
I.17 Generalized Eigenvalues and Eigenvectors 278
I.18 Exercises 280
II Miscellaneous Mathematics 283
II.1 Sets and Set Operations 283
II.1.1 General Set Operations 283
II.1.2 Sets in Euclidean Spaces 284
II.2 Supremum and Infimum 285
II.3 Limsup and Liminf 286
II.4Continuity of Concave and Convex Functions 287
II.5 Compactness 288
II.6 Uniform Continuity 290
II.7 Derivatives of Vector and Matrix Functions 291
II.8 The Mean Value Theorem 294
II.9 Taylor’s Theorem 294
II.10 Optimization 296
Contents xiii
III A Brief Review of Complex Analysis 298
III.1 The Complex Number System 298
III.2 The Complex Exponential Function 301
III.3 The Complex Logarithm 303
III.4Series Expansion of the Complex Logarithm 303
III.5 Complex Integration 305
IV Tables of Critical Values 306
References 315
Index 317