<P>
<BR></P>
<P>INTRODUCTION<BR>TO THE MATHEMATICAL AND<BR>STATISTICAL FOUNDATIONS<BR>OF ECONOMETRICS<BR>HERMAN J. BIERENS<BR> <BR>Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, S&atilde;o Paulo<BR>Cambridge University Press<BR>The Edinburgh Building, Cambridge , UK<BR>First published in print format<BR>- ----<BR>- ----<BR>- ----<BR>&copy; Herman J. Bierens 2005<BR>Information on this title: <a href="http://www.cambridge.org/9780521834315" target="_blank" >www.cambridge.org/9780521834315</A><BR>This book is in copyright. Subject to statutory exception and to the provision of<BR>relevant collective licensing agreements, no reproduction of any part may take place<BR>without the written permission of Cambridge University Press.<BR>- ---<BR>- ---<BR>- ---<BR>Cambridge University Press has no responsibility for the persistence or accuracy of<BR>s for external or third-party internet websites referred to in this book, and does not<BR>guarantee that any content on such websites is, or will remain, accurate or appropriate.<BR>Published in the United States of America by Cambridge University Press, New York<BR><a href="http://www.cambridge.org" target="_blank" >www.cambridge.org</A><BR>hardback<BR>paperback<BR>paperback<BR>eBook (NetLibrary)<BR>eBook (NetLibrary)<BR>hardback</P>
<P>Contents<BR>Preface page xv<BR>1 Probability and Measure 1<BR>1.1 The Texas Lotto 1<BR>1.1.1 Introduction 1<BR>1.1.2 Binomial Numbers 2<BR>1.1.3 Sample Space 3<BR>1.1.4Algebras and Sigma-Algebras of Events 3<BR>1.1.5 Probability Measure 4<BR>1.2 Quality Control 6<BR>1.2.1 Sampling without Replacement 6<BR>1.2.2 Quality Control in Practice 7<BR>1.2.3 Sampling with Replacement 8<BR>1.2.4Limits of the Hypergeometric and Binomial<BR>Probabilities 8<BR>1.3 Why Do We Need Sigma-Algebras of Events ? 10<BR>1.4Proper ties of Algebras and Sigma-Algebras 11<BR>1.4.1 General Properties 11<BR>1.4.2 Borel Sets 14<BR>1.5 Properties of Probability Measures 15<BR>1.6 The Uniform Probability Measure 16<BR>1.6.1 Introduction 16<BR>1.6.2 Outer Measure 17<BR>1.7 Lebesgue Measure and Lebesgue Integral 19<BR>1.7.1 Lebesgue Measure 19<BR>1.7.2 Lebesgue Integral 19<BR>1.8 Random Variables and Their Distributions 20<BR>1.8.1 Random Variables and Vectors 20<BR>1.8.2 Distribution Functions 23<BR>1.9 Density Functions 25</P>
<P>1.10 Conditional Probability, Bayes’ Rule,<BR>and Independence 27<BR>1.10.1 Conditional Probability 27<BR>1.10.2 Bayes’ Rule 27<BR>1.10.3 Independence 28<BR>1.11 Exercises 30<BR>Appendix 1.A – Common Structure of the Proofs of Theorems<BR>1.6 and 1.10 32<BR>Appendix 1.B – Extension of an Outer Measure to a<BR>Probability Measure 32<BR>2 Borel Measurability, Integration, and Mathematical<BR>Expectations 37<BR>2.1 Introduction 37<BR>2.2 Borel Measurability 38<BR>2.3 Integrals of Borel-Measurable Functions with Respect<BR>to a Probability Measure 42<BR>2.4General Measurability and Integrals of Random<BR>Variables with Respect to Probability Measures 46<BR>2.5 Mathematical Expectation 49<BR>2.6 Some Useful Inequalities Involving Mathematical<BR>Expectations 50<BR>2.6.1 Chebishev’s Inequality 51<BR>2.6.2 Holder’s Inequality 51<BR>2.6.3 Liapounov’s Inequality 52<BR>2.6.4Mink owski’s Inequality 52<BR>2.6.5 Jensen’s Inequality 52<BR>2.7 Expectations of Products of Independent Random<BR>Variables 53<BR>2.8 Moment-Generating Functions and Characteristic<BR>Functions 55<BR>2.8.1 Moment-Generating Functions 55<BR>2.8.2 Characteristic Functions 58<BR>2.9 Exercises 59<BR>Appendix 2.A – Uniqueness of Characteristic Functions 61<BR>3 Conditional Expectations 66<BR>3.1 Introduction 66<BR>3.2 Properties of Conditional Expectations 72<BR>3.3 Conditional Probability Measures and Conditional<BR>Independence 79<BR>3.4Conditioning on Increasing Sigma-Algebras 80</P>
<P>3.5 Conditional Expectations as the Best Forecast Schemes 80<BR>3.6 Exercises 82<BR>Appendix 3.A – Proof of Theorem 3.12 83<BR>4 Distributions and Transformations 86<BR>4.1 Discrete Distributions 86<BR>4.1.1 The Hypergeometric Distribution 86<BR>4.1.2 The Binomial Distribution 87<BR>4.1.3 The Poisson Distribution 88<BR>4.1.4 The Negative Binomial Distribution 88<BR>4.2 Transformations of Discrete Random Variables and<BR>Vectors 89<BR>4.3 Transformations of Absolutely Continuous Random<BR>Variables 90<BR>4.4 Transformations of Absolutely Continuous Random<BR>Vectors 91<BR>4.4.1 The Linear Case 91<BR>4.4.2 The Nonlinear Case 94<BR>4.5 The Normal Distribution 96<BR>4.5.1 The Standard Normal Distribution 96<BR>4.5.2 The General Normal Distribution 97<BR>4.6 Distributions Related to the Standard Normal<BR>Distribution 97<BR>4.6.1 The Chi-Square Distribution 97<BR>4.6.2 The Student’s t Distribution 99<BR>4.6.3 The Standard Cauchy Distribution 100<BR>4.6.4 The F Distribution 100<BR>4.7 The Uniform Distribution and Its Relation to the<BR>Standard Normal Distribution 101<BR>4.8 The Gamma Distribution 102<BR>4.9 Exercises 102<BR>Appendix 4.A – Tedious Derivations 104<BR>Appendix 4.B – Proof of Theorem 4.4 106<BR>5 The Multivariate Normal Distribution and Its Application<BR>to Statistical Inference 110<BR>5.1 Expectation and Variance of Random Vectors 110<BR>5.2 The Multivariate Normal Distribution 111<BR>5.3 Conditional Distributions of Multivariate Normal<BR>Random Variables 115<BR>5.4Independence of Linear and Quadratic Transformations<BR>of Multivariate Normal Random Variables 117</P>
<P>5.5 Distributions of Quadratic Forms of Multivariate<BR>Normal Random Variables 118<BR>5.6 Applications to Statistical Inference under Normality 119<BR>5.6.1 Estimation 119<BR>5.6.2 Confidence Intervals 122<BR>5.6.3 Testing Parameter Hypotheses 125<BR>5.7 Applications to Regression Analysis 127<BR>5.7.1 The Linear Regression Model 127<BR>5.7.2 Least-Squares Estimation 127<BR>5.7.3 Hypotheses Testing 131<BR>5.8 Exercises 133<BR>Appendix 5.A – Proof of Theorem 5.8 134<BR>6 Modes of Convergence 137<BR>6.1 Introduction 137<BR>6.2 Convergence in Probability and the Weak Law of Large<BR>Numbers 140<BR>6.3 Almost-Sure Convergence and the Strong Law of Large<BR>Numbers 143<BR>6.4The Uniform Law of Large Numbers and Its<BR>Applications 145<BR>6.4.1 The Uniform Weak Law of Large Numbers 145<BR>6.4.2 Applications of the Uniform Weak Law of<BR>Large Numbers 145<BR>6.4.2.1 Consistency of M-Estimators 145<BR>6.4.2.2 Generalized Slutsky’s Theorem 148<BR>6.4.3 The Uniform Strong Law of Large Numbers<BR>and Its Applications 149<BR>6.5 Convergence in Distribution 149<BR>6.6 Convergence of Characteristic Functions 154<BR>6.7 The Central Limit Theorem 155<BR>6.8 Stochastic Boundedness, Tightness, and the Op and op<BR>Notations 157<BR>6.9 Asymptotic Normality of M-Estimators 159<BR>6.10 Hypotheses Testing 162<BR>6.11 Exercises 163<BR>Appendix 6.A – Proof of the Uniform Weak Law of<BR>Large Numbers 164<BR>Appendix 6.B – Almost-Sure Convergence and Strong Laws of<BR>Large Numbers 167<BR>Appendix 6.C – Convergence of Characteristic Functions and<BR>Distributions 174</P>
<P>7 Dependent Laws of Large Numbers and Central Limit<BR>Theorems 179<BR>7.1 Stationarity and the Wold Decomposition 179<BR>7.2 Weak Laws of Large Numbers for Stationary Processes 183<BR>7.3 Mixing Conditions 186<BR>7.4Unifor m Weak Laws of Large Numbers 187<BR>7.4.1 Random Functions Depending on<BR>Finite-Dimensional Random Vectors 187<BR>7.4.2 Random Functions Depending on<BR>Infinite-Dimensional Random Vectors 187<BR>7.4.3 Consistency of M-Estimators 190<BR>7.5 Dependent Central Limit Theorems 190<BR>7.5.1 Introduction 190<BR>7.5.2 A Generic Central Limit Theorem 191<BR>7.5.3 Martingale Difference Central Limit Theorems 196<BR>7.6 Exercises 198<BR>Appendix 7.A – Hilbert Spaces 199<BR>8 Maximum Likelihood Theory 205<BR>8.1 Introduction 205<BR>8.2 Likelihood Functions 207<BR>8.3 Examples 209<BR>8.3.1 The Uniform Distribution 209<BR>8.3.2 Linear Regression with Normal Errors 209<BR>8.3.3 Probit and Logit Models 211<BR>8.3.4The Tobit Model 212<BR>8.4Asymptotic Properties of ML Estimators 214<BR>8.4.1 Introduction 214<BR>8.4.2 First- and Second-Order Conditions 214<BR>8.4.3 Generic Conditions for Consistency and<BR>Asymptotic Normality 216<BR>8.4.4 Asymptotic Normality in the Time Series Case 219<BR>8.4.5 Asymptotic Efficiency of the ML Estimator 220<BR>8.5 Testing Parameter Restrictions 222<BR>8.5.1 The Pseudo t-Test and the Wald Test 222<BR>8.5.2 The Likelihood Ratio Test 223<BR>8.5.3 The Lagrange Multiplier Test 225<BR>8.5.4Selecting a Test 226<BR>8.6 Exercises 226<BR>I Review of Linear Algebra 229<BR>I.1 Vectors in a Euclidean Space 229<BR>I.2 Vector Spaces 232</P>
<P>I.3 Matrices 235<BR>I.4The Inverse and Transpose of a Matrix 238<BR>I.5 Elementary Matrices and Permutation Matrices 241<BR>I.6 Gaussian Elimination of a Square Matrix and the<BR>Gauss–Jordan Iteration for Inverting a Matrix 244<BR>I.6.1 Gaussian Elimination of a Square Matrix 244<BR>I.6.2 The Gauss–Jordan Iteration for Inverting a<BR>Matrix 248<BR>I.7 Gaussian Elimination of a Nonsquare Matrix 252<BR>I.8 Subspaces Spanned by the Columns and Rows<BR>of a Matrix 253<BR>I.9 Projections, Projection Matrices, and Idempotent<BR>Matrices 256<BR>I.10 Inner Product, Orthogonal Bases, and Orthogonal<BR>Matrices 257<BR>I.11 Determinants: Geometric Interpretation and<BR>Basic Properties 260<BR>I.12 Determinants of Block-Triangular Matrices 268<BR>I.13 Determinants and Cofactors 269<BR>I.14In verse of a Matrix in Terms of Cofactors 272<BR>I.15 Eigenvalues and Eigenvectors 273<BR>I.15.1 Eigenvalues 273<BR>I.15.2 Eigenvectors 274<BR>I.15.3 Eigenvalues and Eigenvectors of Symmetric<BR>Matrices 275<BR>I.16 Positive Definite and Semidefinite Matrices 277<BR>I.17 Generalized Eigenvalues and Eigenvectors 278<BR>I.18 Exercises 280<BR>II Miscellaneous Mathematics 283<BR>II.1 Sets and Set Operations 283<BR>II.1.1 General Set Operations 283<BR>II.1.2 Sets in Euclidean Spaces 284<BR>II.2 Supremum and Infimum 285<BR>II.3 Limsup and Liminf 286<BR>II.4Continuity of Concave and Convex Functions 287<BR>II.5 Compactness 288<BR>II.6 Uniform Continuity 290<BR>II.7 Derivatives of Vector and Matrix Functions 291<BR>II.8 The Mean Value Theorem 294<BR>II.9 Taylor’s Theorem 294<BR>II.10 Optimization 296</P>
<P>III A Brief Review of Complex Analysis 298<BR>III.1 The Complex Number System 298<BR>III.2 The Complex Exponential Function 301<BR>III.3 The Complex Logarithm 303<BR>III.4Series Expansion of the Complex Logarithm 303<BR>III.5 Complex Integration 305<BR>IV Tables of Critical Values 306<BR>References 315<BR>Index 317</P>