Wide Neural Networks of Any Depth Evolve as
     Linear Models Under Gradient Descent

       Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri
         Roman Novak, Jascha Sohl-Dickstein, Jeffrey Pennington
                        Google Brain
     {jaehlee, xlc, schsam, yasamanb, romann, jaschasd, jpennin}@google.com

                        Abstract
     A longstanding goal in deep learning research has been to precisely characterize
     training and generalization. However, th ...