Contents

Preface xiii

Chapter 1 Introduction: The Bayesian Method, its Benefits and Implementation 1

1.1 The Bayes approach and its potential advantages 1

1.2 Expressing prior uncertainty about parameters and Bayesian updating 2

1.3 MCMC sampling and inferences from posterior densities 5

1.4 The main MCMC sampling algorithms 9

1.4.1 Gibbs sampling 12

1.5 Convergence of MCMC samples 14

1.6 Predictions from sampling: using the posterior predictive density 18

1.7 The present book 18

References 19

Chapter 2 Bayesian Model Choice, Comparison and Checking 25

2.1 Introduction: the formal approach to Bayes model choice and averaging 25

2.2 Analytic marginal likelihood approximations and the Bayes information criterion 28

2.3 Marginal likelihood approximations from the MCMC output 30

2.4 Approximating Bayes factors or model probabilities 36

2.5 Joint space search methods 38

2.6 Direct model averaging by binary and continuous selection indicators 41

2.7 Predictive model comparison via cross-validation 43

2.8 Predictive fit criteria and posterior predictive model checks 46

2.9 The DIC criterion 48

2.10 Posterior and iteration-specific comparisons of likelihoods and penalised likelihoods 50

2.11 Monte carlo estimates of model probabilities 52

References 57

Chapter 3 The Major Densities and their Application 63

3.1 Introduction 63

3.2 Univariate normal with known variance 64

3.3 Inference on univariate normal parameters, mean and variance unknown 69

3.4 Heavy tailed and skew density alternatives to the normal 71

3.5 Categorical distributions: binomial and binary data 74

3.5.1 Simulating controls through historical exposure 76

3.6 Poisson distribution for event counts 79

3.7 The multinomial and dirichlet densities for categorical and proportional data 82

3.8 Multivariate continuous data: multivariate normal and t densities 85

3.8.1 Partitioning multivariate priors 87

3.8.2 The multivariate t density 88

3.9 Applications of standard densities: classification rules 91

3.10 Applications of standard densities: multivariate discrimination 98

Exercises 100

References 102

Chapter 4 Normal Linear Regression, General Linear Models and Log-Linear Models 109

4.1 The context for Bayesian regression methods 109

4.2 The normal linear regression model 111

4.2.1 Unknown regression variance 112

4.3 Normal linear regression: variable and model selection, outlier detection and error form 116

4.3.1 Other predictor and model search methods 118

4.4 Bayesian ridge priors for multicollinearity 121

4.5 General linear models 123

4.6 Binary and binomial regression 123

4.6.1 Priors on regression coefficients 124

4.6.2 Model checks 126

4.7 Latent data sampling for binary regression 129

4.8 Poisson regression 132

4.8.1 Poisson regression for contingency tables 134

4.8.2 Log-linear model selection 139

4.9 Multivariate responses 140

Exercises 143

References 146

Chapter 5 Hierarchical Priors for Pooling Strength and Overdispersed Regression Modelling 151

5.1 Hierarchical priors for pooling strength and in general linear model regression 151

5.2 Hierarchical priors: conjugate and non-conjugate mixing 152

5.3 Hierarchical priors for normal data with applications in meta-analysis 153

5.4 Pooling strength under exchangeable models for poisson outcomes 157

5.4.1 Hierarchical prior choices 158

5.4.2 Parameter sampling 159

5.5 Combining information for binomial outcomes 162

5.6 Random effects regression for overdispersed count and binomial data 165

5.7 Overdispersed normal regression: the scale-mixture student model 169

5.8 The normal meta-analysis model allowing for heterogeneity in study design or patient risk 173

5.9 Hierarchical priors for multinomial data 176

5.9.1 Histogram smoothing 177

Exercises 179

References 183

Chapter 6 Discrete Mixture Priors 187

6.1 Introduction: the relevance and applicability of discrete mixtures 187

6.2 Discrete mixtures of parametric densities 188

6.2.1 Model choice 190

6.3 Identifiability constraints 191

6.4 Hurdle and zero-inflated models for discrete data 195

6.5 Regression mixtures for heterogeneous subpopulations 197

6.6 Discrete mixtures combined with parametric random effects 200

6.7 Non-parametric mixture modelling via dirichlet process priors 201

6.8 Other non-parametric priors 207

Exercises 212

References 216

Chapter 7 Multinomial and Ordinal Regression Models 219

7.1 Introduction: applications with categoric and ordinal data 219

7.2 Multinomial logit choice models 221

7.3 The multinomial probit representation of interdependent choices 224

7.4 Mixed multinomial logit models 228

7.5 Individual level ordinal regression 230

7.6 Scores for ordered factors in contingency tables 235

Exercises 237

References 238

Chapter 8 Time Series Models 241

8.1 Introduction: alternative approaches to time series models 241

8.2 Autoregressive models in the observations 242

8.2.1 Priors on autoregressive coefficients 244

8.2.2 Initial conditions as latent data 246

8.3 Trend stationarity in the AR1 model 248

8.4 Autoregressive moving average models 250

8.5 Autoregressive errors 253

8.6 Multivariate series 255

8.7 Time series models for discrete outcomes 257

8.7.1 Observation-driven autodependence 257

8.7.2 INAR models 258

8.7.3 Error autocorrelation 259

8.8 Dynamic linear models and time varying coefficients 261

8.8.1 Some common forms of DLM 264

8.8.2 Priors for time-specific variances or interventions 267

8.8.3 Nonlinear and non-Gaussian state-space models 268

8.9 Models for variance evolution 273

8.9.1 ARCH and GARCH models 274

8.9.2 Stochastic volatility models 275

8.10 Modelling structural shifts and outliers 277

8.10.1 Markov mixtures and transition functions 279

8.11 Other nonlinear models 282

Exercises 285

References 288

Chapter 9 Modelling Spatial Dependencies 297

9.1 Introduction: implications of spatial dependence 297

9.2 Discrete space regressions for metric data 298

9.3 Discrete spatial regression with structured and unstructured random effects 303

9.3.1 Proper CAR priors 306

9.4 Moving average priors 311

9.5 Multivariate spatial priors and spatially varying regression effects 313

9.6 Robust models for discontinuities and non-standard errors 317

9.7 Continuous space modelling in regression and interpolation 321

Exercises 325

References 329

Chapter 10 Nonlinear and Nonparametric Regression 333

10.1 Approaches to modelling nonlinearity 333

10.2 Nonlinear metric data models with known functional form 335

10.3 Box–Cox transformations and fractional polynomials 338

10.4 Nonlinear regression through spline and radial basis functions 342

10.4.1 Shrinkage models for spline coefficients 345

10.4.2 Modelling interaction effects 346

10.5 Application of state-space priors in general additive nonparametric regression 350

10.5.1 Continuous predictor space prior 351

10.5.2 Discrete predictor space priors 353

Exercises 359

References 362

Chapter 11 Multilevel and Panel Data Models 367

11.1 Introduction: nested data structures 367

11.2 Multilevel structures 369

11.2.1 The multilevel normal linear model 369

11.2.2 General linear mixed models for discrete outcomes 370

11.2.3 Multinomial and ordinal multilevel models 372

11.2.4 Robustness regarding cluster effects 373

11.2.5 Conjugate approaches for discrete data 374

11.3 Heteroscedasticity in multilevel models 379

11.4 Random effects for crossed factors 381

11.5 Panel data models: the normal mixed model and extensions 387

11.5.1 Autocorrelated errors 390

11.5.2 Autoregression in y 391

11.6 Models for panel discrete (binary, count and categorical) observations 393

11.6.1 Binary panel data 393

11.6.2 Repeated counts 395

11.6.3 Panel categorical data 397

11.7 Growth curve models 400

11.8 Dynamic models for longitudinal data: pooling strength over units and times 403

11.9 Area apc and spatiotemporal models 407

11.9.1 Age–period data 408

11.9.2 Area–time data 409

11.9.3 Age–area–period data 409

11.9.4 Interaction priors 410

Exercises 413

References 418

Chapter 12 Latent Variable and Structural Equation Models for Multivariate Data 425

12.1 Introduction: latent traits and latent classes 425

12.2 Factor analysis and SEMS for continuous data 427

12.2.1 Identifiability constraints in latent trait (factor analysis) models 429

12.3 Latent class models 433

12.3.1 Local dependence 437

12.4 Factor analysis and SEMS for multivariate discrete data 441

12.5 Nonlinear factor models 447

Exercises 450

References 452

Chapter 13 Survival and Event History Analysis 457

13.1 Introduction 457

13.2 Parametric survival analysis in continuous time 458

13.2.1 Censored observations 459

13.2.2 Forms of parametric hazard and survival curves 460

13.2.3 Modelling covariate impacts and time dependence in the hazard rate 461

13.3 Accelerated hazard parametric models 464

13.4 Counting process models 466

13.5 Semiparametric hazard models 469

13.5.1 Priors for the baseline hazard 470

13.5.2 Gamma process prior on cumulative hazard 472

13.6 Competing risk-continuous time models 475

13.7 Variations in proneness: models for frailty 477

13.8 Discrete time survival models 482

Exercises 486

References 487

Chapter 14 Missing Data Models 493

14.1 Introduction: types of missingness 493

14.2 Selection and pattern mixture models for the joint data-missingness density 494

14.3 Shared random effect and common factor models 498

14.4 Missing predictor data 500

14.5 Multiple imputation 503

14.6 Categorical response data with possible non-random missingness: hierarchical and regression models 506

14.6.1 Hierarchical models for response and non-response by strata 506

14.6.2 Regression frameworks 510

14.7 Missingness with mixtures of continuous and categorical data 516

14.8 Missing cells in contingency tables 518

14.8.1 Ecological inference 519

Exercises 526

References 529

Chapter 15 Measurement Error, Seemingly Unrelated Regressions, and Simultaneous Equations 533

15.1 Introduction 533

15.2 Measurement error in both predictors and response in normal linear regression 533

15.2.1 Prior information on X or its density 535

15.2.2 Measurement error in general linear models 537

15.3 Misclassification of categorical variables 541

15.4 Simultaneous equations and instruments for endogenous variables 546

15.5 Endogenous regression involving discrete variables 550

Exercises 554

References 556

Appendix 1 A Brief Guide to Using WINBUGS 561

A1.1 Procedure for compiling and running programs 561

A1.2 Generating simulated data 562

A1.3 Other advice 563

Index 565