On the Ineffectiveness of Variance Reduced
       Optimization for Deep Learning

                 Aaron Defazio & Leon Bottou
                 Facebook AI Research New York

                       Abstract
     The application of stochastic variance reduction to optimization has shown re-
     markable recent theoretical and practical success. The applicability of these tech-
     niques to the hard non-convex optimization problems encountered during training
     of modern deep neural network ...