Algebraic Geometry and Statistical Learning Theory
Algebraic Geometry and Statistical
Learning Theory
SUMIO WATANABE
Tokyo Institute of Technology



Preface page vii
1 Introduction 1
1.1 Basic concepts in statistical learning 1
1.2 Statistical models and learning machines 10
1.3 Statistical estimation methods 18
1.4 Four main formulas 26
1.5 Overview of this book 41
1.6 Probability theory 42
2 Singularity theory 48
2.1 Polynomials and analytic functions 48
2.2 Algebraic set and analytic set 50
2.3 Singularity 53
2.4 Resolution of singularities 58
2.5 Normal crossing singularities 66
2.6 Manifold 72
3 Algebraic geometry 77
3.1 Ring and ideal 77
3.2 Real algebraic set 80
3.3 Singularities and dimension 86
3.4 Real projective space 87
3.5 Blow-up 91
3.6 Examples 99
4 Zeta function and singular integral 105
4.1 Schwartz distribution 105
4.2 State density function 111
v
4.3 Mellin transform 116
4.4 Evaluation of singular integral 118
4.5 Asymptotic expansion and b-function 128
5 Empirical processes 133
5.1 Convergence in law 133
5.2 Function-valued analytic functions 140
5.3 Empirical process 144
5.4 Fluctuation of Gaussian processes 154
6 Singular learning theory 158
6.1 Standard form of likelihood ratio function 160
6.2 Evidence and stochastic complexity 168
6.3 Bayes and Gibbs estimation 177
6.4 Maximum likelihood and a posteriori 203
7 Singular learning machines 217
7.1 Learning coefficient 217
7.2 Three-layered neural networks 227
7.3 Mixture models 230
7.4 Bayesian network 233
7.5 Hidden Markov model 234
7.6 Singular learning process 235
7.7 Bias and variance 239
7.8 Non-analytic learning machines 245
8 Singular statistics 249
8.1 Universally optimal learning 249
8.2 Generalized Bayes information criterion 252
8.3 Widely applicable information criteria 253
8.4 Singular hypothesis test 258
8.5 Realization of a posteriori distribution 264
8.6 From regular to singular 274
Bibliography 277
Index 284