Bootstrap Methods: A Guide for Practitioners and Researchers
Second Edition
MICHAEL R. CHERNICK
United BioSource Corporation
Newtown, PA
1. What Is Bootstrapping? 1
1.1. Background, 1
1.2. Introduction, 8
  1.3.  Wide Range of Applications,  13
1.4. Historical Notes, 16
1.5. Summary, 24
2. Estimation 26
2.1. Estimating Bias, 26
    2.1.1.  How to Do It by Bootstrapping,  26
    2.1.2.  Error Rate Estimation in Discrimination,  28
    2.1.3.  Error Rate Estimation: An Illustrative Problem,  39
    2.1.4.  Efron’s Patch Data Example,  44
  2.2.  Estimating Location and Dispersion,  46
   2.2.1. Means and Medians, 47
    2.2.2.  Standard Errors and Quartiles,  48
2.3. Historical Notes, 51
3. Confi  dence Sets and Hypothesis Testing 53
3.1. Confi  dence Sets,  55
    3.1.1.  Typical Value Theorems for M-Estimates,  55
   3.1.2. Percentile Method, 57
    3.1.3.  Bias Correction and the Acceleration Constant,  58
   3.1.4. Iterated Bootstrap, 61
   3.1.5. Bootstrap Percentile t Confi  dence Intervals,  64
  3.2.   Relationship Between Confi  dence Intervals and Tests of
Hypotheses, 64
3.3. Hypothesis Testing Problems, 66
    3.3.1.  Tendril DX Lead Clinical Trial Analysis,  67
  3.4.   An Application of Bootstrap Confi  dence Intervals to Binary
Dose–Response Modeling,  71
3.5.  Historical Notes, 75
4. Regression Analysis 78
4.1.  Linear Models, 82
   4.1.1.  Gauss–Markov Theory, 83
    4.1.2.   Why Not Just Use Least Squares?  83
    4.1.3.   Should I Bootstrap the Residuals from the Fit?  84
4.2.  Nonlinear Models, 86
    4.2.1.   Examples of Nonlinear Models,  87
    4.2.2.   A Quasi-optical Experiment,  89
4.3.  Nonparametric Models, 93
4.4.  Historical Notes, 94
5. Forecasting and Time Series Analysis 97
  5.1.   Methods of Forecasting,  97
  5.2.   Time Series Models,  98
  5.3.   When Does Bootstrapping Help with Prediction Intervals?  99
  5.4.   Model-Based Versus Block Resampling,  103
  5.5.   Explosive Autoregressive Processes,  107
  5.6.   Bootstrapping-Stationary Arma Models,  108
5.7.  Frequency-Based Approaches, 108
5.8.  Sieve Bootstrap, 110
5.9.  Historical Notes, 111
6. Which Resampling Method Should You Use? 114
6.1.  Related Methods, 115
   6.1.1.  Jackknife, 115
    6.1.2.   Delta Method, Infi  nitesimal Jackknife, and Infl  uence
Functions, 116
   6.1.3.  Cross-Validation, 119
   6.1.4.  Subsampling, 119
6.2.  Bootstrap Variants, 120
   6.2.1.  Bayesian Bootstrap, 121
    6.2.2.   The Smoothed Boostrap,  123
    6.2.3.   The Parametric Bootstrap,  124
   6.2.4.  Double Bootstrap, 125
    6.2.5.   The m-out-of-n Bootstrap,  125
7. Effi  cient and Effective Simulation 127
  7.1.   How Many Replications?  128
  7.2.   Variance Reduction Methods,  129
   7.2.1.  Linear Approximation, 129
   7.2.2.  Balanced Resampling, 131
   7.2.3.  Antithetic Variates, 132
   7.2.4.  Importance Sampling, 133
   7.2.5.  Centering, 134
  7.3.   When Can Monte Carlo Be Avoided?  135
7.4.  Historical Notes, 136
8. Special Topics 1 3 9
8.1.  Spatial Data, 139
   8.1.1.  Kriging, 139
    8.1.2.   Block Bootstrap on Regular Grids,  142
    8.1.3.   Block Bootstrap on Irregular Grids,  143
8.2.  Subset Selection, 143
  8.3.   Determining the Number of Distributions in a Mixture
Model, 145
8.4.  Censored Data, 148
8.5.   p-Value Adjustment,  149
    8.5.1.   Description of Westfall–Young Approach,  150
    8.5.2.   Passive Plus DX Example,  150
   8.5.3.  Consulting Example, 152
8.6.  Bioequivalence Applications, 153
   8.6.1.  Individual Bioequivalence, 153
   8.6.2.  Population Bioequivalence, 155
  8.7.   Process Capability Indices,  156
8.8.  Missing Data, 164
8.9.  Point Processes, 166
8.10.  Lattice Variables, 168
8.11.  Historical Notes, 169
9. When Bootstrapping Fails Along with Remedies for Failures 172
  9.1.   Too Small of a Sample Size,  173
  9.2.   Distributions with Infi  nite Moments,  175
   9.2.1.  Introduction, 175
    9.2.2.   Example of Inconsistency,  176
   9.2.3.  Remedies, 176
  9.3.   Estimating Extreme Values,  177
   9.3.1.  Introduction, 177
   9.3.2. Example of Inconsistency, 177
   9.3.3. Remedies, 178
9.4.  Survey Sampling, 179
   9.4.1.  Introduction, 179
   9.4.2. Example of Inconsistency, 180
   9.4.3. Remedies, 180
  9.5.   Data Sequences that Are M-Dependent, 180
   9.5.1.  Introduction, 180
    9.5.2.   Example of Inconsistency When Independence Is
Assumed, 181
   9.5.3. Remedies, 181
  9.6.   Unstable Autoregressive Processes,  182   9.6.1.  Introduction, 182
    9.6.2.   Example of Inconsistency,  182
   9.6.3.  Remedies, 183
9.7.  Long-Range Dependence, 183
   9.7.1.  Introduction, 183
    9.7.2.   Example of Inconsistency,  183
   9.7.3.  Remedies, 184
9.8.  Bootstrap Diagnostics, 184
9.9.  Historical Notes, 185
Bibliography 1 (Prior to 1999) 188
Bibliography 2 (1999–2007) 274
Author Index 330
Subject Index 359