Root Mean Square Layer Normalization

                 Biao Zhang1 Rico Sennrich2,1
             1
              School of Informatics, University of Edinburgh
          2
           Institute of Computational Linguistics, University of Zurich
              B.Zhang@ed.ac.uk, sennrich@cl.uzh.ch

                       Abstract
     Layer normalization (LayerNorm) has been successfully applied to various deep
     neural networks to help stabilize training and boost model convergence because
      ...