全部版块 我的主页
论坛 数据科学与人工智能 数据分析与数据科学 SPSS论坛
3526 2
2014-04-13

Is it possible to penalize coefficients toward a number other than zero in a ridge regression in R?

For example, let's say I have dependent variable Y and independent variables X1,X2,X3, and X4. Because of the multicollinear nature of the ivs, ridge regression is appropriate. But say I'm fairly certain that the coefficient of X1 is near 5, X2 is near 1, X3 is near -1, and X4 is near -5.

Is there a ridge package and method in R where I can implement penalties on the coefficients of the ivs that penalize them toward those numbers instead of 0? I'd love to see an example in R with my example data, if possible. Thank you.


二维码

扫码加我 拉你入群

请注明:姓名-公司-职位

以便审核进群资格,未注明则拒绝

全部回复
2014-4-13 05:44:06
A simple way to do this is to subtract the "centering value" of the coefficient times its associated variable from the left-hand side. To go with your example,

Y=β1X1+β2X2+β3X3+β4X4+e
Assume the coefficient values should be centered at (5,1,-1,-5) respectively. Then:

Y−5X1−X2+X3+5X4=(β1−5)X1+(β2−1)X2+(β3+1)X3+(β4+5)X4+e
and, redefining terms, you have:

Y∗=β∗1X1+β∗2X2+β∗3X3+β∗4X4+e
A standard ridge regression would shrink the β∗i towards 0, which is equivalent to shrinking the original βi towards the specified centering values. to see this, consider a fully-shrunk β∗4=0, then β4+5=0 and therefore β4=−5. Shrinkage accomplished!
二维码

扫码加我 拉你入群

请注明:姓名-公司-职位

以便审核进群资格,未注明则拒绝

2014-4-13 05:45:09
Ridge regression sets a normal prior centered at zero for each parameter. To penalize coefficients towards different values, just center the priors around your target instead of around 0. I think you can accomplish this using the bayesGLM function of the arm package with the following parameters:

prior.mean : set this to vector containing the values you want to use as the centers for your priors (the values you want to penalize "towards")
prior.df : set this to Inf to get normal priors. Possibly not necessary. There should be a way to control your prior variance, but it's not clear to me how this would be accomplished from looking at the documentation.
http://cran.r-project.org/web/packages/arm/arm.pdf
二维码

扫码加我 拉你入群

请注明:姓名-公司-职位

以便审核进群资格,未注明则拒绝

相关推荐
栏目导航
热门文章
推荐文章

说点什么

分享

扫码加好友,拉您进群
各岗位、行业、专业交流群