摘要翻译:
由于Markov链Monte Carlo(MCMC)需要应用大量参数,且参数的数量随阶数的增加而迅速增加,所以高阶交互作用下的贝叶斯分类和回归在很大程度上是不可行的。在本文中,我们展示了如何通过有效地减少参数的数量,利用许多交互在所有训练案例中具有相同的值这一事实来使其变得可行。我们的方法使用一个“压缩”参数来表示与一组模式相关联的所有参数的总和,这些模式对于所有训练案例都具有相同的值。利用对称稳定分布作为原始参数的先验值,我们可以很容易地找到这些压缩参数的先验值。因此,当用MCMC训练模型时,我们只需要处理数量少得多的压缩参数。在考虑最高可能的阶数之前,压缩参数的数目可能已经收敛。在训练完模型后,我们可以根据需要将这些压缩参数拆分为原始参数,以便对测试用例进行预测。我们详细介绍了如何压缩logistic序列预测模型的参数。对模拟数据和实际数据的实验表明,我们的压缩方法确实可以减少大量的参数。
---
英文标题:
《A Method for Compressing Parameters in Bayesian Models with Application
to Logistic Sequence Prediction Models》
---
作者:
Longhai Li and Radford M. Neal
---
最新提交年份:
2007
---
分类信息:
一级分类:Statistics 统计学
二级分类:Machine Learning
机器学习
分类描述:Covers machine learning papers (supervised, unsupervised, semi-supervised learning, graphical models, reinforcement learning, bandits, high dimensional inference, etc.) with a statistical or theoretical grounding
覆盖机器学习论文(监督,无监督,半监督学习,图形模型,强化学习,强盗,高维推理等)与统计或理论基础
--
一级分类:Statistics 统计学
二级分类:Methodology 方法论
分类描述:Design, Surveys, Model Selection, Multiple Testing, Multivariate Methods, Signal and Image Processing, Time Series, Smoothing, Spatial Statistics, Survival Analysis, Nonparametric and Semiparametric Methods
设计,调查,模型选择,多重检验,多元方法,信号和图像处理,时间序列,平滑,空间统计,生存分析,非参数和半参数方法
--
---
英文摘要:
Bayesian classification and regression with high order interactions is largely infeasible because Markov chain Monte Carlo (MCMC) would need to be applied with a great many parameters, whose number increases rapidly with the order. In this paper we show how to make it feasible by effectively reducing the number of parameters, exploiting the fact that many interactions have the same values for all training cases. Our method uses a single ``compressed'' parameter to represent the sum of all parameters associated with a set of patterns that have the same value for all training cases. Using symmetric stable distributions as the priors of the original parameters, we can easily find the priors of these compressed parameters. We therefore need to deal only with a much smaller number of compressed parameters when training the model with MCMC. The number of compressed parameters may have converged before considering the highest possible order. After training the model, we can split these compressed parameters into the original ones as needed to make predictions for test cases. We show in detail how to compress parameters for logistic sequence prediction models. Experiments on both simulated and real data demonstrate that a huge number of parameters can indeed be reduced by our compression method.
---
PDF链接:
https://arxiv.org/pdf/711.4983