Descending through a Crowded Valley —
               Benchmarking Deep Learning Optimizers

               Robin M. Schmidt * 1 Frank Schneider * 1 Philipp Hennig 1 2

             Abstract                1. Introduction
                                 Large-scale stochastic optimization drives a wide variety of
   Choosing the optimizer is considered to be among      machine learning tasks. Because choosing the right opti-
   the most crucial design decisions in deep learning,     mization met ...