摘要翻译：
我们定义{em预测信息}$I_{rm pred}(T)$为时间序列过去和未来之间的相互信息。在大的观察次数$T$的极限中，发现了三种性质不同的行为:$I_{\rm pred}(T)$可以保持有限，以对数方式增长，或以分数幂律增长。如果时间序列允许我们学习具有有限个参数的模型，那么$I_{\rm pred}(T)$与计算模型空间维数的系数成对数增长。相反，幂律增长与学习无限参数（或非参数）模型有关，例如具有光滑性约束的连续函数。在学习理论和通过统计力学和动力系统理论对物理系统的分析中都定义了预测信息和复杂性度量之间的联系。此外，正如熵提供了与一些简单而合理的条件一致的可用信息的唯一测度一样，我们认为$I_{\rm pred}(T)$的发散部分提供了时间序列背后动力学复杂性的唯一测度。最后，我们讨论了这些思想在物理、统计学和生物学中的不同问题中是如何有用的。
---
英文标题：
《Predictability, complexity and learning》
---
作者：
William Bialek, Ilya Nemenman, and Naftali Tishby
---
最新提交年份：
2001
---
分类信息：

一级分类：Physics        物理学
二级分类：Data Analysis, Statistics and Probability        数据分析、统计与概率
分类描述：Methods, software and hardware for physics data analysis: data processing and storage; measurement methodology; statistical and mathematical aspects such as parametrization and uncertainties.
物理数据分析的方法、软硬件：数据处理与存储；测量方法；统计和数学方面，如参数化和不确定性。
--
一级分类：Physics        物理学
二级分类：Disordered Systems and Neural Networks        无序系统与神经网络
分类描述：Glasses and spin glasses; properties of random, aperiodic and quasiperiodic systems; transport in disordered media; localization; phenomena mediated by defects and disorder; neural networks
眼镜和旋转眼镜；随机、非周期和准周期系统的性质；无序介质中的传输；本地化；由缺陷和无序介导的现象；神经网络
--
一级分类：Physics        物理学
二级分类：Other Condensed Matter        其他凝聚态物质
分类描述：Work in condensed matter that does not fit into the other cond-mat classifications
在不适合其他cond-mat分类的凝聚态物质中工作
--
一级分类：Computer Science        计算机科学
二级分类：Machine Learning        机器学习
分类描述：Papers on all aspects of machine learning research (supervised, unsupervised, reinforcement learning, bandit problems, and so on) including also robustness, explanation, fairness, and methodology. cs.LG is also an appropriate primary category for applications of machine learning methods.
关于机器学习研究的所有方面的论文（有监督的，无监督的，强化学习，强盗问题，等等），包括健壮性，解释性，公平性和方法论。对于机器学习方法的应用，CS.LG也是一个合适的主要类别。
--
一级分类：Physics        物理学
二级分类：Adaptation and Self-Organizing Systems        自适应和自组织系统
分类描述：Adaptation, self-organizing systems, statistical physics, fluctuating systems, stochastic processes, interacting particle systems, machine learning
自适应，自组织系统，统计物理，波动系统，随机过程，相互作用粒子系统，机器学习
--
一级分类：Quantitative Biology        数量生物学
二级分类：Other Quantitative Biology        其他定量生物学
分类描述：Work in quantitative biology that does not fit into the other q-bio classifications
不适合其他q-bio分类的定量生物学工作
--

---
英文摘要：
  We define {\em predictive information} $I_{\rm pred} (T)$ as the mutual information between the past and the future of a time series. Three qualitatively different behaviors are found in the limit of large observation times $T$: $I_{\rm pred} (T)$ can remain finite, grow logarithmically, or grow as a fractional power law. If the time series allows us to learn a model with a finite number of parameters, then $I_{\rm pred} (T)$ grows logarithmically with a coefficient that counts the dimensionality of the model space. In contrast, power--law growth is associated, for example, with the learning of infinite parameter (or nonparametric) models such as continuous functions with smoothness constraints. There are connections between the predictive information and measures of complexity that have been defined both in learning theory and in the analysis of physical systems through statistical mechanics and dynamical systems theory. Further, in the same way that entropy provides the unique measure of available information consistent with some simple and plausible conditions, we argue that the divergent part of $I_{\rm pred} (T)$ provides the unique measure for the complexity of dynamics underlying a time series. Finally, we discuss how these ideas may be useful in different problems in physics, statistics, and biology.
---
PDF链接：
https://arxiv.org/pdf/physics/0007070