Contents

1 An Introduction to Sample Surveys and Nonresponse 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Classical sampling theory . . . . . . . . . . . . . . . . . . . . . . 2

1.2.1 Sample surveys . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.2 Sampling design . . . . . . . . . . . . . . . . . . . . . . . 5

1.2.3 Estimators using auxiliary information . . . . . . . . . . . 6

1.3 Surveys in practice . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3.1 Sources of error . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3.2 Missing data in surveys . . . . . . . . . . . . . . . . . . . 10

1.3.3 Di®erent response types . . . . . . . . . . . . . . . . . . . 12

1.3.4 Implications of unit nonresponse . . . . . . . . . . . . . . 14

1.3.5 The importance of auxiliary information . . . . . . . . . . 19

1.4 Overview of the thesis . . . . . . . . . . . . . . . . . . . . . . . . 20

Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

1.A The Horvitz-Thompson estimator . . . . . . . . . . . . . . . . . . 24

1.B The generalized regression estimator . . . . . . . . . . . . . . . . 25

1.C The calibration framework . . . . . . . . . . . . . . . . . . . . . . 27

2 Causes and Correlates of Survey Nonresponse 29

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.2 Causes of contact and participation . . . . . . . . . . . . . . . . . 31

2.2.1 Contact . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.2.2 Survey participation . . . . . . . . . . . . . . . . . . . . . 32

2.3 Correlates of contact and participation . . . . . . . . . . . . . . . 35

2.3.1 Literature review . . . . . . . . . . . . . . . . . . . . . . . 35

2.3.2 Contact . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.3.3 Survey participation . . . . . . . . . . . . . . . . . . . . . 38

i

ii Contents

2.4 Classifying sample elements . . . . . . . . . . . . . . . . . . . . . 41

2.4.1 Classes of participants and continuum of resistance . . . . 41

2.4.2 A multi-dimensional continuum based on response prob-

abilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.5 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 44

3 Analysis of Nonresponse in the Dutch Labour Force Survey 47

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.2 Aspects of the Labour Force Survey . . . . . . . . . . . . . . . . 48

3.2.1 Sampling design . . . . . . . . . . . . . . . . . . . . . . . 48

3.2.2 Data collection . . . . . . . . . . . . . . . . . . . . . . . . 49

3.2.3 Objective of the LFS . . . . . . . . . . . . . . . . . . . . . 50

3.2.4 Response rates . . . . . . . . . . . . . . . . . . . . . . . . 51

3.2.5 Fieldwork strategy . . . . . . . . . . . . . . . . . . . . . . 51

3.3 Data for analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.3.1 ELFS sample . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.3.2 Linked data . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.3.3 Unit of analysis . . . . . . . . . . . . . . . . . . . . . . . . 60

3.4 Nonresponse analysis . . . . . . . . . . . . . . . . . . . . . . . . . 61

3.4.1 Model selection and interpretation . . . . . . . . . . . . . 61

3.4.2 The process propensity . . . . . . . . . . . . . . . . . . . 63

3.4.3 The contact propensity . . . . . . . . . . . . . . . . . . . 65

3.4.4 The propensity for being able to participate . . . . . . . . 69

3.4.5 The participation propensity . . . . . . . . . . . . . . . . 71

3.4.6 Contrast with more simpli¯ed response processes . . . . . 75

3.4.7 Summary of the models . . . . . . . . . . . . . . . . . . . 80

3.5 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 83

4 Re-Approaching Nonrespondents 85

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

4.2 Design of the re-approach strategies . . . . . . . . . . . . . . . . 88

4.2.1 General design of the LFS pilot . . . . . . . . . . . . . . . 88

4.2.2 The call-back approach . . . . . . . . . . . . . . . . . . . 89

4.2.3 The basic-question approach . . . . . . . . . . . . . . . . 91

4.3 Analysis of the response in the re-approach strategies . . . . . . . 100

4.3.1 The call-back approach . . . . . . . . . . . . . . . . . . . 100

4.3.2 The basic-question approach . . . . . . . . . . . . . . . . 105

4.3.3 Summary of the results . . . . . . . . . . . . . . . . . . . 109

4.4 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 112

Contents iii

5 The R-Indicator As a Supplement to the Response Rate 115

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

5.2 The concept of representative response . . . . . . . . . . . . . . . 118

5.2.1 The meaning of representative . . . . . . . . . . . . . . . 118

5.2.2 De¯nition of a representative subsample of respondents . 120

5.3 R-indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

5.3.1 Population based R-indicator . . . . . . . . . . . . . . . . 121

5.3.2 Sample based R-indicator . . . . . . . . . . . . . . . . . . 124

5.4 Features of an R-indicator . . . . . . . . . . . . . . . . . . . . . . 125

5.4.1 Features in general . . . . . . . . . . . . . . . . . . . . . . 126

5.4.2 Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . 127

5.4.3 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . 129

5.4.4 Response-representativeness functions . . . . . . . . . . . 131

5.5 An empirical validation of the sample-based R-indicator . . . . . 132

5.5.1 Standard error and con¯dence interval . . . . . . . . . . . 133

5.5.2 Re-approaching nonrespondents to the Dutch LFS . . . . 134

5.5.3 Mixed mode pilots . . . . . . . . . . . . . . . . . . . . . . 136

5.6 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 140

Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

5.A Estimating response probabilities . . . . . . . . . . . . . . . . . . 143

6 A Review of Nonresponse Adjustment Methods 145

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

6.2 The generalized regression estimator . . . . . . . . . . . . . . . . 149

6.3 Calibration estimators . . . . . . . . . . . . . . . . . . . . . . . . 151

6.4 The propensity score method . . . . . . . . . . . . . . . . . . . . 155

6.4.1 Two-phase approach . . . . . . . . . . . . . . . . . . . . . 155

6.4.2 The propensity score method in internet panels . . . . . . 157

6.4.3 The response propensity method . . . . . . . . . . . . . . 158

6.5 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 161

7 Adjustment for Undercoverage and Nonresponse 165

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

7.2 Description of the data . . . . . . . . . . . . . . . . . . . . . . . . 168

7.3 The methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

7.3.1 Telephone coverage propensity . . . . . . . . . . . . . . . 170

7.3.2 Simultaneous adjustment of undercoverage and nonresponse173

7.4 Analysis of the telephone POLS survey . . . . . . . . . . . . . . . 174

7.4.1 Nonresponse adjustment . . . . . . . . . . . . . . . . . . . 175

iv Contents

7.4.2 Undercoverage adjustment . . . . . . . . . . . . . . . . . . 177

7.4.3 Simultaneous adjustment . . . . . . . . . . . . . . . . . . 179

7.4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

7.5 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 182

8 Analysis and Adjustment Methods with Di®erent Response

Types 183

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

8.2 Nonresponse analysis models . . . . . . . . . . . . . . . . . . . . 186

8.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 186

8.2.2 The nested logit model . . . . . . . . . . . . . . . . . . . . 187

8.2.3 Bivariate probit model with sample selection . . . . . . . 190

8.2.4 Bivariate probit analysis of nonresponse to the Dutch LFS 194

8.2.5 A comparison of the bivariate probit model with sample

selection and the nested logit model . . . . . . . . . . . . 196

8.2.6 Multilevel model . . . . . . . . . . . . . . . . . . . . . . . 197

8.2.7 Advantages and disadvantages . . . . . . . . . . . . . . . 202

8.3 Alternative methods for nonresponse adjustment . . . . . . . . . 203

8.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 203

8.3.2 Sequential weight adjustment method . . . . . . . . . . . 205

8.3.3 The sample selection model . . . . . . . . . . . . . . . . . 207

8.3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

8.4 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 221

9 Nonresponse Adjustment in Mixed Mode Surveys 223

9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

9.2 Data collection modes . . . . . . . . . . . . . . . . . . . . . . . . 224

9.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 224

9.2.2 Mode di®erences . . . . . . . . . . . . . . . . . . . . . . . 228

9.2.3 Mode e®ects . . . . . . . . . . . . . . . . . . . . . . . . . 234

9.3 Mixed mode designs . . . . . . . . . . . . . . . . . . . . . . . . . 241

9.3.1 Instrument design . . . . . . . . . . . . . . . . . . . . . . 242

9.3.2 Concurrent and sequential design . . . . . . . . . . . . . . 242

9.4 Combining data from di®erent data collection modes . . . . . . . 244

9.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 244

9.4.2 Paradata . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

9.5 Nonresponse adjustment methods in mixed mode surveys . . . . 249

9.5.1 Concurrent mixed mode and nonresponse adjustment . . 249

9.5.2 Sequential mixed mode and nonresponse adjustment . . . 259

9.6 Application to the pilot Safety Monitor-1 . . . . . . . . . . . . . 264

9.6.1 Survey items in the Safety Monitor . . . . . . . . . . . . . 264

9.6.2 Mixed mode approaches applied to the pilot SM-1 . . . . 266

9.6.3 Summary of the models and the results . . . . . . . . . . 272

9.7 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 276

10 Summary and Conclusions 279