Contents
Preface ix
About the Authors xi
Foreword xiii
Prologue xv
Part I Fundamentals 1
1 Sampling Plans 3
1.1 The ‘Curse of Dimensionality’ and How to Avoid It 4
1.2 Physical versus Computational Experiments 4
1.3 Designing Preliminary Experiments (Screening) 6
1.3.1 Estimating the Distribution of Elementary Effects 6
1.4 Designing a Sampling Plan 13
1.4.1 Stratification 13
1.4.2 Latin Squares and Random Latin Hypercubes 15
1.4.3 Space-filling Latin Hypercubes 17
1.4.4 Space-filling Subsets 28
1.5 A Note on Harmonic Responses 29
1.6 Some Pointers for Further Reading 30
References 31
2 Constructing a Surrogate 33
2.1 The Modelling Process 33
2.1.1 Stage One: Preparing the Data and Choosing a Modelling Approach 33
2.1.2 Stage Two: Parameter Estimation and Training 35
2.1.3 Stage Three: Model Testing 36
2.2 Polynomial Models 40
2.2.1 Example One: Aerofoil Drag 42
2.2.2 Example Two: a Multimodal Testcase 44
2.2.3 What About the k-variable Case? 45
vi Contents
2.3 Radial Basis Function Models 45
2.3.1 Fitting Noise-Free Data 45
2.3.2 Radial Basis Function Models of Noisy Data 49
2.4 Kriging 49
2.4.1 Building the Kriging Model 51
2.4.2 Kriging Prediction 59
2.5 Support Vector Regression 63
2.5.1 The Support Vector Predictor 64
2.5.2 The Kernel Trick 67
2.5.3 Finding the Support Vectors 68
2.5.4 Finding  70
2.5.5 Choosing C and  71
2.5.6 Computing : -SVR 73
2.6 The Big(ger) Picture 75
References 76
3 Exploring and Exploiting a Surrogate 77
3.1 Searching the Surrogate 78
3.2 Infill Criteria 79
3.2.1 Prediction Based Exploitation 79
3.2.2 Error Based Exploration 84
3.2.3 Balanced Exploitation and Exploration 85
3.2.4 Conditional Likelihood Approaches 91
3.2.5 Other Methods 101
3.3 Managing a Surrogate Based Optimization Process 102
3.3.1 Which Surrogate for What Use? 102
3.3.2 How Many Sample Plan and Infill Points? 102
3.3.3 Convergence Criteria 103
3.4 Search of the Vibration Isolator Geometry Feasibility Using
Kriging Goal Seeking 104
References 106
Part II Advanced Concepts 109
4 Visualization 111
4.1 Matrices of Contour Plots 112
4.2 Nested Dimensions 114
Reference 116
5 Constraints 117
5.1 Satisfaction of Constraints by Construction 117
5.2 Penalty Functions 118
5.3 Example Constrained Problem 121
5.3.1 Using a Kriging Model of the Constraint Function 121
5.3.2 Using a Kriging Model of the Objective Function 123
5.4 Expected Improvement Based Approaches 125
5.4.1 Expected Improvement With Simple Penalty Function 126
5.4.2 Constrained Expected Improvement 126
5.5 Missing Data 131
5.5.1 Imputing Data for Infeasible Designs 133
Contents vii
5.6 Design of a Helical Compression Spring Using Constrained
Expected Improvement 136
5.7 Summary 139
References 139
6 Infill Criteria with Noisy Data 141
6.1 Regressing Kriging 143
6.2 Searching the Regression Model 144
6.2.1 Re-Interpolation 146
6.2.2 Re-Interpolation With Conditional Likelihood Approaches 149
6.3 A Note on Matrix Ill-Conditioning 152
6.4 Summary 152
References 153
7 Exploiting Gradient Information 155
7.1 Obtaining Gradients 155
7.1.1 Finite Differencing 155
7.1.2 Complex Step Approximation 156
7.1.3 Adjoint Methods and Algorithmic Differentiation 156
7.2 Gradient-enhanced Modelling 157
7.3 Hessian-enhanced Modelling 162
7.4 Summary 165
References 165
8 Multi-fidelity Analysis 167
8.1 Co-Kriging 167
8.2 One-variable Demonstration 173
8.3 Choosing Xc and Xe 176
8.4 Summary 177
References 177
9 Multiple Design Objectives 179
9.1 Pareto Optimization 179
9.2 Multi-objective Expected Improvement 182
9.3 Design of the Nowacki Cantilever Beam Using Multi-objective, Constrained
Expected Improvement 186
9.4 Design of a Helical Compression Spring Using Multi-objective, Constrained
Expected Improvement 191
9.5 Summary 192
References 192
Appendix: Example Problems 195
A.1 One-Variable Test Function 195
A.2 Branin Test Function 196
A.3 Aerofoil Design 197
A.4 The Nowacki Beam 198
A.5 Multi-objective, Constrained Optimal Design of a Helical Compression Spring 200
A.6 Novel Passive Vibration Isolator Feasibility 202
References 203
Index 205