Contents

Preface ix

1. Introduction 1

1.1 Two Examples, 3

1.1.1 Public School Class Sizes, 4

1.1.2 Value at Risk, 5

1.2 Observables, Unobservables, and Objects of Interest, 7

1.3 Conditioning and Updating, 10

1.4 Simulators, 13

1.5 Modeling, 15

1.6 Decisionmaking, 17

2. Elements of Bayesian Inference 21

2.1 Basics, 21

2.2 Sufficiency, Ancillarity, and Nuisance Parameters, 31

2.2.1 Sufficiency, 31

2.2.2 Ancillarity, 33

2.2.3 Nuisance Parameters, 35

2.3 Conjugate Prior Distributions, 38

2.4 Bayesian Decision Theory and Point Estimation, 46

2.5 Credible Sets, 56

2.6 Model Comparison, 61

2.6.1 Marginal Likelihoods, 62

3. Topics in Bayesian Inference 73

3.1 Hierarchical Priors and Latent Variables, 73

3.2 Improper Prior Distributions, 78

3.3 Prior Robustness and the Density Ratio Class, 87

3.4 Asymptotic Analysis, 91

3.5 The Likelihood Principle, 97

4. Posterior Simulation 105

4.1 Direct Sampling, 106

4.2 Acceptance and Importance Sampling, 110

4.2.1 Acceptance Sampling, 111

4.2.2 Importance Sampling, 114

4.3 Markov Chain Monte Carlo, 119

4.3.1 The Gibbs Sampler, 120

4.3.2 The Metropolis–Hastings Algorithm, 122

4.4 Variance Reduction, 127

4.4.1 Concentrated Expectations, 128

4.4.2 Antithetic Sampling, 130

4.5 Some Continuous State Space Markov Chain Theory, 133

4.5.1 Convergence of the Gibbs Sampler, 137

4.5.2 Convergence of the Metropolis–Hastings Algorithm, 139

4.6 Hybrid Markov Chain Monte Carlo Methods, 142

4.6.1 Transition Mixtures, 142

4.6.2 Metropolis within Gibbs, 143

4.7 Numerical Accuracy and Convergence in Markov Chain Monte Carlo, 145

5. Linear Models 153

5.1 BACC and the Normal Linear Regression Model, 154

5.2 Seemingly Unrelated Regressions Models, 162

5.3 Linear Constraints in the Linear Model, 169

5.3.1 Linear Inequality Constraints, 170

5.3.2 Conjectured Linear Restrictions, Linear Inequality Constraints, and Covariate Selection, 172

5.4 Nonlinear Regression, 175

5.4.1 Nonlinear Regression with Smoothness Priors, 176

6. Modeling with Latent Variables 195

6.1 Censored Normal Linear Models, 196

6.2 Probit Linear Models, 200

6.3 The Independent Finite State Model, 202

6.4 Modeling with Mixtures of Normal Distributions, 205

6.4.1 The Independent Student-t Linear Model, 206

6.4.2 Normal Mixture Linear Models, 208

6.4.3 Generalizing the Observable Outcomes, 215

7. Modeling for Time Series 221

7.1 Linear Models with Serial Correlation, 222

7.2 The First-Order Markov Finite State Model, 226

7.2.1 Inference in the Nonstationary Model, 229

7.2.2 Inference in the Stationary Model, 230

7.3 Markov Normal Mixture Linear Model, 233

8. Bayesian Investigation 245

8.1 Implementing Simulation Methods, 246

8.1.1 Density Ratio Tests, 247

8.1.2 Joint Distribution Tests, 251

8.2 Formal Model Comparison, 255

8.2.1 Bayes Factors for Modeling with Common Likelihoods, 255

8.2.2 Marginal Likelihood Approximation Using Importance Sampling, 256

8.2.3 Marginal Likelihood Approximation Using Gibbs Sampling, 257

8.2.4 Density Ratio Marginal Likelihood Approximation, 259

8.3 Model Specification, 262

8.3.1 Prior Predictive Analysis, 262

8.3.2 Posterior Predictive Analysis, 267

8.4 Bayesian Communication, 271

8.5 Density Ratio Robustness Bounds, 277

Bibliography 283

Author Index 293