Tensor Programs I: Wide Feedforward or Recurrent
Neural Networks of Any Architecture are Gaussian
           Processes

                      Greg Yang
                    Microsoft Research AI
                   gregyang@microsoft.com

                       Abstract
     Wide neural networks with random weights and biases are Gaussian processes,
     as observed by Neal (1995) for shallow networks, and more recently by Lee et
     al. (2018) and Matthews et al. (2018) for deep fully-connected ...