Andrew Gelman & Adam Zelizer, 2014; Evidence on the deleterious impact of sustained useof polynomial regression on causal inference | Feb, 21st


  "It is common in regression discontinuity (RD) analysis to control for third- or fifth-degree polynomialsof the assignment variable. Such models can overfit, leading to causal inferences thatare substantively implausible and that arbitrarily attribute variation to the high-degree polynomialor the discontinuity. This paper examines two recent studies that make use of regressiondiscontinuity to discuss evident practical problems with these estimates and how they interactwith pathologies of the current system of scientific publication. First, we discuss a recent studythat estimates the effect on air pollution and life expectancy of a coal-heating policy in China (Chen, Y., Ebenstein, A., Greenstone, M., and Li, H.; 2013).The reported effects, based on a third-degree polynomial, are statistically significant but substantivelydubious, and are sensitive to model choice. This study is indicative of a category ofpolicy analyses where strong claims are based on weak data and methodologies which permit theresearcher wide latitude in presenting estimated treatment effects. We then replicate a procedurefrom Green et al., in which regression discontinuity is used to recover estimated treatmenteffects relative to an experimental benchmark, to illustrate one practical problem with the RDestimates in the coal-heating paper: high-degree polynomials yield noisy estimates of treatmenteffects that do not accurately convey uncertainty. We recommend that (a) researchers considerthe problems which may result from controlling for higher-order polynomials; and (b) that journalsrecognize that quantitative analyses of policy issues are often inconclusive and relax theimplicit rule under which statistical significance is a condition for publication. "


据说这是一篇八卦，以及，讽刺：

...Speculations are presented as fact. For example, the China air pollution study was featured in a New York Times article (Wong, 2013) that referred unquestioningly to “the 5.5-year drop in life expectancy in the north,” as well as in a New Yorker article by a Pulitzer prizewinning reporter (Johnson, 2013) who simply wrote that a study “noted that pollution from coal reduces average life expectancy in northern China by five and a half years,” with no indication that the “five and a half years” number was just a point estimate, even setting aside questions about the validity of that estimate.
--------------------