Springer Texts in Statistics
1 Introduction 1
1.1 WhatIsNonparametricInference? ................ 1
1.2 NotationandBackground..................... 2
1.3 Confidence Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 UsefulInequalities ......................... 8
1.5 BibliographicRemarks....................... 10
1.6 Exercises .............................. 10
2 Estimating the cdf and
Statistical Functionals 13
2.1 The cdf ............................... 13
2.2 EstimatingStatisticalFunctionals ................ 15
2.3 Influence Functions . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4 Empirical Probability Distributions . . . . . . . . . . . . . . . . 21
2.5 BibliographicRemarks....................... 23
2.6 Appendix .............................. 23
2.7 Exercises .............................. 24
3 The Bootstrap and the Jackknife 27
3.1 TheJackknife............................ 27
3.2 TheBootstrap ........................... 30
3.3 ParametricBootstrap ....................... 31
3.4 Bootstrap Confidence Intervals . . . . . . . . . . . . . . . . . . 32
3.5 SomeTheory ............................ 35
3.6 BibliographicRemarks....................... 37
3.7 Appendix .............................. 37
3.8 Exercises .............................. 39
Smoothing: General Concepts 43
4.1 TheBias–VarianceTradeoff.................... 50
4.2 Kernels ............................... 55
4.3 WhichLossFunction? ....................... 57
4.4 Confidence Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.5 TheCurseofDimensionality ................... 58
4.6 BibliographicRemarks....................... 59
4.7 Exercises .............................. 59
Nonparametric Regression 61
5.1 ReviewofLinearandLogisticRegression ............ 63
5.2 LinearSmoothers.......................... 66
5.3 ChoosingtheSmoothingParameter ............... 68
5.4 LocalRegression .......................... 71
5.5 Penalized Regression, Regularization and Splines . . . . . . . . 81
5.6 VarianceEstimation ........................ 85
5.7 Confidence Bands . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.8 AverageCoverage.......................... 94
5.9 SummaryofLinearSmoothing .................. 95
5.10 Local Likelihood and Exponential Families . . . . . . . . . . . . 96
5.11Scale-SpaceSmoothing....................... 99
5.12MultipleRegression ........................100
5.13OtherIssues.............................111
5.14BibliographicRemarks.......................119
5.15Appendix ..............................119
5.16Exercises ..............................120
Density Estimation 125
6.1 Cross-Validation ..........................126
6.2 Histograms .............................127
6.3 KernelDensityEstimation.....................131
6.4 LocalPolynomials .........................137
6.5 MultivariateProblems .......................138
6.6 Converting Density Estimation Into Regression . . . . . . . . . 139
6.7 BibliographicRemarks.......................140
6.8 Appendix ..............................140
6.9 Exercises ..............................142
Normal Means and Minimax Theory 145
7.1 TheNormalMeansModel.....................145
7.2 FunctionSpaces...........................147
7.3 Connection to Regression and Density Estimation . . . . . . . 149
7.4 Stein’s Unbiased Risk Estimator (sure) .............150
7.5 MinimaxRiskandPinsker’sTheorem ..............153
7.6 Linear Shrinkage and the James–Stein Estimator . . . . . . . . 155
7.7 Adaptive Estimation Over Sobolev Spaces . . . . . . . . . . . . 158
7.8 Confidence Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
7.9 Optimality of Confidence Sets . . . . . . . . . . . . . . . . . . . 166
7.10RandomRadiusBands? ......................170
7.11Penalization,OraclesandSparsity ................171
7.12BibliographicRemarks.......................172
7.13Appendix ..............................173
7.14Exercises ..............................180
Nonparametric Inference Using Orthogonal Functions 183
8.1 Introduction.............................183
8.2 NonparametricRegression.....................183
8.3 IrregularDesigns ..........................190
8.4 DensityEstimation.........................192
8.5 ComparisonofMethods ......................193
8.6 TensorProductModels ......................193
8.7 BibliographicRemarks.......................194
8.8 Exercises ..............................194
Wavelets and Other Adaptive Methods 197
9.1 HaarWavelets ...........................199
9.2 ConstructingWavelets.......................203
9.3 WaveletRegression.........................206
9.4 WaveletThresholding .......................208
9.5 BesovSpaces ............................211
9.6 Confidence Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
9.7 Boundary Corrections and Unequally Spaced Data . . . . . . . 215
9.8 OvercompleteDictionaries.....................215
9.9 OtherAdaptiveMethods .....................216
9.10DoAdaptiveMethodsWork? ...................220
9.11BibliographicRemarks.......................221
9.12Appendix ..............................221
9.13Exercises ..............................223
0 Other Topics 227
10.1MeasurementError.........................227
10.2InverseProblems ..........................233
10.3NonparametricBayes........................235
10.4SemiparametricInference .....................235
10.5CorrelatedErrors..........................236
10.6Classification ............................236
10.7Sieves ................................237
10.8Shape-RestrictedInference.....................237
10.9Testing ...............................238
10.10ComputationalIssues .......................240
10.11Exercises ..............................240
Bibliography 243
List of Symbols 259
Table of Distributions 261
Index 263