A User's Guide To Principal Components
J. EDWARD JACKSON
A Wiley-Interscienc Publication
JOHN WILEY & SONS, INC.
New York • Chichester • Brisbane • Toronto · Singapore


Contents
Preface XV
Introduction I
I. Getting Started 4
1.1 Introduction, 4
1.2 A Hypothetical Example, 4
1.3 Characteristic Roots and Vectors, 7
1.4 The Method of Principal Components, 10
1.5 Some Propeties of Princial Components, 13
1.6 Scaling of Characteristic Vectors, 16
1.7 Using Principal Components in Quality Control, 19
l. PCA With More Than Two Variables 26
2.1 Introduction, 26
2.2 Sequential Estimation of Principal Components, 27
2.3 Ballistic Missile Example, 28
2.4 Covarianc Matrice of Less than Full Rank, 30
2.5 Characteristic Roots are Equal or Nearly So, 32
2.6 A Test for Equality of Roots, 33
2.7 Residual Analysis, 34
2.8 When to Stop?, 41
2.9 A Photographic Film Example, 51
2.10 Uses of PCA, 58
3. Scling of Data 63
3.1 Introduction, 63
3.2 Data as Deviations from the Mean: Covarianc
Matrices, 64
3.3 Data in Standard Units: Correlation Matrice, 6
vii
viii CONTENTS
3.4 Data are not Scaled at All: Product or Second Moment
Matrice, 72
3.5 Double-cetered Matrices, 75
3.6 Weighted PCA, 75
3.7 Complex Variables, 77
4. Inferential Procedures
4.1 Introduction, 80
4.2 Sampling Properties of Characteristic Roots and
Vectors, 80
4.3 Optimality, 85
4.4 Tests for Equality of Characteristic Roots, 86
4.5 Distribution of Characteristic Roots, 89
4.6 Signifianc Tests for Characteristic Vectors:
Confimatory PCA, 95
4.7 Inferenc with Regard to Correlation Matrice, 98
4.8 The Effct of Nonnormality, 102
4.9 The Complex Domain, 104
5. Putting It All Together-Hearing Loss I
5.1 Introduction, 105
5.2 The Data, 106
5.3 Principal Component Analysis, 110
5.4 Data Analysis, 115
6. Opeations with Group Data
6.1 Introduction, 123
6.2 Rational Subgroups and Generalized T-statistics, 123
6.3 Generalized T-statistics Using PCA, 126
6.4 Generalized Residual Analysis, 128
6.5 Use of Hypothetical or Sample Means and Covariance
Matrice, 131
6.6 Numerical Example: A Color Film Proces, 132
6.7 Generalized T-statistics and the Multivariate Analysis of
Variance 141
7. Vector Interpretation I:
Simplifiations and Inferential Techniques
7.1 Introduction, 142
7.2 Interpretation. Some General Rules, 143
80
105
123
142
CONTENTS ix
7.3 Simplifiation, 144
7.4 Use of Confimatory PCA, 148
7.5 Correlation of Vector Coeffients, 149
8. Vector Interpretation II: Rotation 155
8.1 Introduction, 155
8.2 Simple Structure, 156
8.3 Simple Rotation, 157
8.4 Rotation Methods, 159
8.5 Some Comments About Rotation, 165
8.6 Procrustes Rotation, 167
9. A Case History-Hearing Loss II 173
9.1 Introduction, 173
9.2 The Data, 174
9.3 Principal Component Analysis, 177
9.4 Allowance for Age, 178
9.5 Putting it all Together, 184
9.6 Analysis of Groups, 186
10. Singular Value Decomposition:
Multidimensional Scling I 189
10.1 Introduction, 189
10.2 R- and Q-analysis, 189
10.3 Singular Value Decomposition, 193
10.4 Introduction to Multidimensional Scaling, 196
10.5 Biplots, 199
10.6 MDPREF, 204
10.7 Point-Point Plots, 211
10.8 Correspondenc Analysis, 214
10.9 Three-Way PCA, 230
10.10 N-Mode PCA, 232
II. Distance Models:
Multidimensional Scling II 233
11.1 Similarity Models, 233
11.2 An Example, 234
11.3 Data Collection Techniques, 237
11.4 Enhanced MDS Scaling of Similarities, 239
X CONTENTS
11.5 Do Horseshoes Bring Good Luck?, 250
11.6 Scaling Individual Diffrences, 252
11.7 External Analysis of Similarity Spaces, 257
11.8 Other Scaling Techniques, Including One-Dimensional
Scales, 262
12. Liner Models 1: Regression;
PCA of Predictor Variables
12.1 Introduction, 263
12.2 Classical Least Squares, 264
12.3 Principal Components Regression, 271
12.4 Methods Involving Multiple Responses, 281
12.5 Partial Least-Squares Regression, 282
12.6 Redundancy Analysis, 290
12.7 Summary, 298
13. Liner Models II: Analysis of Varianc;
PCA of Response Variables
13.1 Introduction, 301
13.2 Univariate Analysis of Variance, 302
13.3 MANOVA, 303
13.4 Alternative MANOVA using PCA, 305
13.5 Comparison of Methods, 308
13.6 Extension to Other Designs, 309
13.7 An Application of PCA to Univariate ANOVA, 309
14. Other Applications of PCA
14.1 Missing Data, 319
14.2 Using PCA to Improve Data Quality, 324
14.3 Tests for Multivariate Normality, 325
14.4 Variate Selection, 328
14.5 Discriminant Analysis and Cluster Analysis, 334
14.6 Time Series, 338
15. Flatland Speial Proceds for Two Dimensions
15.1 Construction of a Probability Ellipse, 342
15.2 Inferential Procedures for the Orthogonal Regression
Line, 344
15.3 Correlation Matrices, 348
15.4 Reduced Major Axis, 348
263
301
319
342
CONTENTS xi
16. Odds and End 350
16.1 Introduction, 350
16.2 Generalized PCA, 350
16.3 Cross-validation, 353
16.4 Sensitivity, 356
16.5 Robust PCA, 365
16.6 g-Group PCA, 372
16.7 PCA When Data Are Functions, 376
16.8 PCA With Discrete Data, 381
16.9 [Odds and Ends]l, 385
17. What is Factor Analysis Anyhow? 388
17.1 Introduction, 388
17.2 The Factor Analysis Model, 389
17.3 Estimation Methods, 398
17.4 Class I Estimation Proceures, 399
17.5 Class II Estimation Procedures, 402
17.6 Comparison of Estimation Procedures, 405
17.7 Factor Score Estimates, 407
17.8 Confimatory Factor Analysis, 412
17.9 Other Factor Analysis Techniques, 416
17.10 Just What is Factor Analysis Anyhow?, 420
18. Other Competitors 424
18.1 Introduction, 424
18.2 Image Analysis, 425
18.3 Triangularization Methods, 427
18.4 Arbitrary Components, 430
18.5 Subsets of Variables, 430
18.6 Andrews' Function Plots, 432
Concusion 435
Appendix A. Matrix Propeties 437
A.l Introduction, 437
A.2 Defiitions, 437
A.3 Operations with Matrice, 441
Appendix B. Matrix Algebra Assocated with Principal Componet
Analysis 446
xii
Appedx C. Computational Method
C.l Introduction, 450
C.2 Solution of the Characteristic Equation, 450
C.3 The Power Method, 451
C.4 Higher-Level Techniques, 453
C.5 Computer Packages, 454
Appendx D. A Directory of Symbols an Defiitions for PCA
D.l Symbols, 456
D.2 Defiitions, 459
Appendix E. Some Classic Examples
Appendx F.
E.1 Introduction, 460
E.2 Examples for which the Original Data are
Available, 460
E.3 Covariance or Correlation Matrices Only, 462
Data Sets Used in This Boo
F.l Introduction, 464
F.2 Chemical Example, 464
F.3 Grouped Chemical Example, 465
F.4 Ballistic Missile Example, 466
F.5 Black-and-White Film Example, 466
F.6 Color Film Example, 467
F.7 Color Print Example, 467
F.8 Seventh-Grade Tests, 468
F.9 Absorbence Curves, 468
F.lO Complex Variables Example, 468
F.l1 Audiometric Example, 469
F.l2 Audiometric Case History, 470
F.13 Rotation Demonstration, 470
F.14 Physical Measurements, 470
F.15 Rectangular Data Matrix, 470
F.16 Horseshoe Example, 471
F.l7 Presidential Hopeuls, 471
F.l8 Contingency Table Demo: Brand vs. Sex, 472
F.l9 Contingency Table Demo: Brand vs. Age, 472
F.20 Three-Way Contingency Table, 472
CONTENTS
450
456
460
464
CONTENTS xiii
F.21 Occurrenc ,of Personal Assault, 472
F.22 Linnerud Data, 473
F.23 Bivariate Nonnormal Distribution, 473
F.24 Circle Data, 473
F.25 United States Budget, 474
Appedix G. Tables 475
G.l Table of the Normal Distribution, 476
G.2 Table of the t-Distribution, 477
G.3 Table of the Chi-square Distribution, 478
G.4 Table of the F-Distribution, 480
G.5 Table of the Lawley-Hotelling Trac
Statistic, 485
G.6 Tables of the Extreme Roots of a Covarianc
Matrix, 494
Bibliography 497
Author Indx 551
Subject Index 563