1.1 Galton's “Shrinkage" Interpretation of Regression
1.2 The Primary “Multivariate Analysis" Theme of This Book
1.3 How are Shrinkage Regression Methods Typically Applied?
1.4 Which Part of the Book Should I Read Next?
2. BASIC LINEAR MODEL CONCEPTS
2.1 Centered Variables
2.2 The Special Case of UNCORRELATED Regressors
2.3 Canonical Form of Regressors
2.4 NUMERICAL versus STATISTICAL ILL-CONDITIONING
2.5 Eigen Decompositions
2.6 The UNCORRELATED COMPONENTS of LEAST SQUARES
2.7 STATISTICAL SIGNIFICANCE of Uncorrelated Components
2.8 Predictions, Residuals & Linear Reparameterizations that remove Ill-Conditioning
2.9 SIGNAL-to-NOISE Ratios
2.10 The Statistical Distribution of Principal Correlations
2.11 When “Should" Coefficients have “Wrong" Signs?
2.12 Tests of General Linear Hypotheses
2.13 Weighted Residual Analyses
3. SHRINKAGE REGRESSION FUNDAMENTALS
3.1 Moments of Generalized Shrinkage Estimators
3.2 Shrinkage Inflation of the Residual Mean Square
3.3 The Hoerl-Kennard ORDINARY RIDGE REGRESSION Family
3.4 The TWO-PARAMETER GENERALIZED RIDGE Family
3.5 The IMPLICIT INTERCEPT Associated with Shrinkage
3.6 Shrinkage in Models Without an INTERCEPT
3.7 Shrinkage Residual Analyses
4. THE RISK OF SHRINKAGE
4.1 Classical “Optimal" Shrinkage
4.1.1 Diagonal Elements of Mean Squared Error Matrices
4.1.2 MSE Measures Depending Only Upon Diagonal Elements
4.1.3 Weighted Mean Squared Error Measures
4.1.4 The MSE in Specific Directions
4.1.5 Balancing Components of MSE Parallel to and Orthogonal to the Unknown True
Coefficient Vector
4.1.6 Canonical Form for Optimal Shrinkage of a Single Fixed-Effect Coefficient
4.2 Classical “Good" Shrinkage
4.3 Classical “Ultimate" Shrinkage
4.4 Random Coefficient Shrinkage
4.4.1 A Within-Batch and Between-Batch Variation Model
4.4.2 Canonical Form for Optimal Shrinkage of a Single Random-Effect Coefficient
4.5 Summary
5. NORMAL-THEORY MAXIMUM LIKELIHOOD: BLUEs and BLUPs
5.1 Unrestricted Maximum Likelihood and BLUE Theory
5.2 The Likelihood of Mean Squared Error Optimality
5.2.1 Unrestricted Maximum Likelihood Shrinkage: The Cubic Estimator
5.2.2 Maximum Likelihood UNIFORM Shrinkage
5.3 Closed Form Expressions within the 2-Parameter Family
5.3.1 The most-likely-to-be-mse-optimal shrinkage extent, k, for given
shape/curvature.
5.3.2 The most-likely-to-be-mse-optimal shrinkage
5.3.3 The limit as the shrinkage shape/curvature, Q, approaches _.
5.3.4 Large Sample Chi-Squared Tests of MSE-Optimality
5.4 Maximum Likelihood Methods for Mixed Linear Models
5.5 Completely Random Models with a Single Variance Component
5.5.1 Demonstration that BLUP estimates are shrinkage estimates in this case.
5.5.2 Random coefficient maximum likelihood choice of shrinkage extent.
6. RISK (MEAN SQUARED ERROR) ESTIMATION and SIMULATION
6.1 Stein's Unbiased Estimate of Overall Predictive Risk
6.1.1 Contraction Towards a Linear Variety
6.1.2 Minimum Mean Squared Error Estimation of 5#
6.1.3 Stein Contraction Formulas
6.2 Estimates of Shrinkage Risk: Fixed Coefficient Cases
6.2.1 Unbiased Normal-Theory Estimates
6.2.2 Correct-Range Estimates
6.2.3 Shrinkage Factors Minimizing Scaled Risk Estimates
6.2.4 The Estimated Risk in Arbitrary Linear Combinations
6.2.5 Mallows-like Estimates of Predictive Mean-Squared-Error
6.3 Estimates of Shrinkage Risk: Random Coefficient Cases
6.4 Monte-Carlo Risk Simulation
6.4.1 Simulated Risk for Fixed Coefficient Models
6.4.2 Simulated Risk for Random Coefficient Models
6.4.3 Summary of Risk Simulation Results
7. RANDOM COEFFICIENT FORMULATIONS
7.1 Estimation of Random Effects
7.2 Estimation of Variance Components
7.3 Variation Between and Within Production Batches
7.4 Pharmaceutical Stability Models
8. BAYESIAN FORMULATIONS
8.1 Bayesian Conjugate-Normal Linear-Model Formulations
8.2 Bayesian Diagnostic Checking
8.3 More Bayes' Measures of the Extent of Shrinkage
8.4 Nonconjugate Bayes Formulations
8.5 An Empirical Bayes Likelihood Approach
9. COMPUTATIONALLY INTENSE METHODS
9.1 Data Perturbations After the Last Decimal Place and the Perturbation-Limit
9.2 Multivariate Normal Errors-in-Variables Models
9.3 Cross-Validation, Bootstrapping, and Sample Reuse Methods
9.4 Iterative Re-Weighting Methods
10. TOPICS of HISTORICAL INTEREST, HEURISTIC ARGUMENTS, and COMMON
MISCONCEPTIONS
10.1 The Contributions of Hoerl and Kennard
10.2 The Obenchain-Vinod “chain-rule-argument"
10.3 Methods based upon Fictitious Data Augmentation
10.4 Preliminary Test Methods for imposing linear restrictions or detecting
multicollinearity
10.5 Methods for Relaxing Correlations among Coefficients
10.6 Methods Utilizing Estimates of One.
Part Two: Shrinkage Regression
Applications and Implementations
11. TRACE DISPLAYS: THE PSYCHOLOGY OF PERCEPTION
11.1 Multicollinearity Allowance (MCAL) Scaling
11.1.1 Alternative Measures of Shrinkage Extent
MCALœk† trace[ ( XTX