Book Description
This monograph presents mathematical theory of statistical models described by the essentially large number of unknown parameters, comparable with sample size but can also be much larger. In this meaning, the proposed theory can be called "essentially multiparametric". It is developed on the basis of the Kolmogorov asymptotic approach in which sample size increases along with the number of unknown parameters.

This theory opens a way for solution of central problems of multivariate statistics, which up until now have not been solved. Traditional statistical methods based on the idea of an infinite sampling often break down in the solution of real problems, and, dependent on data, can be inefficient, unstable and even not applicable. In this situation, practical statisticians are forced to use various heuristic methods in the hope the will find a satisfactory solution.

Mathematical theory developed in this book presents a regular technique for implementing new, more efficient versions of statistical procedures. Near exact solutions are constructed for a number of concrete multi-dimensional problems: estimation of expectation vectors, regression and discriminant analysis, and for the solution to large systems of empiric linear algebraic equations. It is remarkable that these solutions prove to be not only non-degenerating and always stable, but also near exact within a wide class of populations.

In the conventional situation of small dimension and large sample size these new solutions far surpass the classical, commonly used consistent ones. It can be expected in the near future, for the most part, traditional multivariate statistical software will be replaced by the always reliable and more efficient versions of statistical procedures implemented by the technology described in this book.

This monograph will be of interest to a variety of specialists working with the theory of statistical methods and its applications. Mathematicians would find new classes of urgent problems to be solved in their own regions. Specialists in applied statistics creating statistical packages will be interested in more efficient methods proposed in the book. Advantages of these methods are obvious: the user is liberated from the permanent uncertainty of possible instability and inefficiency and gets algorithms with unimprovable accuracy and guaranteed for a wide class of distributions.

A large community of specialists applying statistical methods to real data will find a number of always stable highly accurate versions of algorithms that will help them to better solve their scientific or economic problems. Students and postgraduates will be interested in this book as it will help them get at the foremost frontier of modern statistical science.

- Presents original mathematical investigations
and open a new branch of mathematical statistics
- Illustrates a technique for developing always stable and efficient versions of multivariate statistical analysis for large-dimensional problems
- Describes the most popular methods some near exact solutions; including algorithms of non-degenerating large-dimensional discriminant and regression analysis

About the Author
Graduated from Physical Faculty of Moscow State University (MSU). In 1961 presented the thesis Investigations in Resonance Theory of Nuclear Reactions and won (the first) the degree of doctor in physics and mathematics at MSU Research Institute of Nuclear Physics. In 2001 presented the thesis Asymptotic Theory of Statistical Analysis of Observations of Increasing Dimension and won (the second, advanced) degree of doctor in physics and mathematics at the Faculty of Calculational Mathematics and Cybernetics of MSU. Since 1970 teaching students at Moscow State Institute of Electronics and Mathematics.

CONTENTS
Foreword xi
Preface xiii
Chapter 1. Introduction: the Development of
Multiparametric Statistics 1
The Stein effect . . . . . . . . . . . . . . . . . . 4
The Kolmogorov Asymptotics . . . . . . . . . . 10
Spectral Theory of Increasing Random
Matrices . . . . . . . . . . . . . . . . . . . . . . 12
ConstructingMultiparametric Procedures . . . . 17
Optimal Solution to Empirical Linear
Equations . . . . . . . . . . . . . . . . . . . . . 19
Chapter 2. Fundamental Problem of Statistics 21
2.1. Shrinkage of SampleMean Vector . . . . . . . . . . 23
Shrinkage for Normal Distributions . . . . . . . 24
Shrinkage for a Wide Class of Distributions . . . 29
Conclusions . . . . . . . . . . . . . . . . . . . . 32
2.2. Shrinkage of Unbiased Estimators . . . . . . . . . . 33
Special Shrinkage of Normal Estimators . . . . . 33
Shrinkage of Arbitrary Unbiased Estimators . . 35
Limit Quadratic Risk of Shrinkage Estimators . 41
Conclusions . . . . . . . . . . . . . . . . . . . . 43
2.3. Shrinkage of Infinite-Dimensional Vectors . . . . . 45
Normal distributions . . . . . . . . . . . . . . . 46
Wide Class of Distributions . . . . . . . . . . . 50
Conclusions . . . . . . . . . . . . . . . . . . . . 54
2.4. Unimprovable Component-Wise Estimation . . . . 56
Estimator for the Density of Parameters . . . . 59
Estimator for the Best Estimating Function . . 63
Chapter 3. Spectral Theory of Large Sample
Covariance Matrices 71
3.1. Spectral Functions of Large Sample Covariance
Matrices . . . . . . . . . . . . . . . . . . . . . . . . 75
GramMatrices . . . . . . . . . . . . . . . . . . . 75
Sample CovarianceMatrices . . . . . . . . . . . 83
Limit Spectra . . . . . . . . . . . . . . . . . . . 88
3.2. Spectral Functions of Infinite Sample Covariance
Matrices . . . . . . . . . . . . . . . . . . . . . . . . 97
Dispersion Equations for Infinite Gram
Matrices . . . . . . . . . . . . . . . . . . . . . . 98
Dispersion Equations for Sample Covariance
Matrices . . . . . . . . . . . . . . . . . . . . . . 103
Limit Spectral Equations . . . . . . . . . . . . . 105
3.3. Normalization of Quality Functions . . . . . . . . . 114
Spectral Functions of Sample Covariance
Matrices . . . . . . . . . . . . . . . . . . . . . . 116
Normal Evaluation of Sample-Dependent
Functionals . . . . . . . . . . . . . . . . . . . . . 117
Conclusions . . . . . . . . . . . . . . . . . . . . 124
Chapter 4. Asymptotically Unimprovable
Solution of Multivariate Problems 127
4.1. Estimators of Large Inverse Covariance Matrices . 129
ProblemSetting . . . . . . . . . . . . . . . . . . 130
Shrinkage for Inverse Covariance Matrices . . . 131
Generalized Ridge Estimators . . . . . . . . . . 133
Asymptotically Unimprovable Estimator . . . . 138
Proofs for Section 4.1 . . . . . . . . . . . . . . . 140
4.2. Matrix Shrinkage Estimators of Expectation Vectors 147
Limit Quadratic Risk for Estimators of Vectors 148
Minimization of the Limit Quadratic Risk . . . 154
Statistics to Approximate Limit Risk . . . . . . 159
Statistics to Approximate the Extremum
Solution . . . . . . . . . . . . . . . . . . . . . . 162
CONTENTS ix
4.3. Multiparametric Sample Linear Regression . . . . . 167
Functionals of RandomGramMatrices . . . . . 171
Functionals in the Regression Problem . . . . . 181
Minimization of Quadratic Risk . . . . . . . . . 186
Special Cases . . . . . . . . . . . . . . . . . . . . 190
Chapter 5. Multiparametric Discriminant
Analysis 193
5.1. Discriminant Analysis of Independent Variables . . 195
A PrioriWeighting of Variables . . . . . . . . . 197
EmpiricalWeighting of Variables . . . . . . . . 200
Minimum Error Probability for Empirical
Weighting . . . . . . . . . . . . . . . . . . . . . 203
Statistics to Estimate Probabilities of Errors . . 207
Contribution of a Small Number of Variables . . 209
Selection of Variables by Threshold . . . . . . . 211
5.2. Discriminant Analysis of Dependent Variables . . . 220
Asymptotical Setting . . . . . . . . . . . . . . . 221
Moments of Generalized Discriminant Function 224
Limit Probabilities of Errors . . . . . . . . . . . 227
Best-in-the-Limit Discriminant Procedure . . . . 231
The Extension to a Wide Class of Distributions 234
Estimating the Error Probability . . . . . . . . 236
Chapter 6. Theory of Solution to High-Order
Systems of Empirical Linear Algebraic Equations 239
6.1. The Best Bayes Solution . . . . . . . . . . . . . . . 240
6.2. Asymptotically Unimprovable Solution . . . . . . . 246
Spectral Functions of Large Gram Matrices . . . 248
Limit Spectral Functions of Gram Matrices . . . 251
Quadratic Risk of Pseudosolutions . . . . . . . . 254
Minimization of the Limit Risk . . . . . . . . . 258
Shrinkage-Ridge Pseudosolution . . . . . . . . . 262
Proofs for Section 6.2 . . . . . . . . . . . . . . . 266
Appendix: Experimental Investigation of Spectral
Functions of Large Sample Covariance Matrices 285
References 303
Index 311