The Heavy-Tail Phenomenon in SGD

                Mert Gürbüzbalaban 1 Umut Simsekli 2 Lingjiong Zhu 3

              Abstract                1. Introduction
                                 The learning problem in neural networks can be expressed as
   In recent years, various notions of capacity and       an instance of the well-known population risk minimization
   complexity have been proposed for characterizing       problem in statistics, given as follows:
   the generalization propertie ...