Piecewise Strong Convexity of Neural Networks

                      Tristan Milne
                   Department of Mathematics
                    University of Toronto
                   Toronto, Ontario, Canada
                  tmilne@math.toronto.edu

                       Abstract
     We study the loss surface of a feed-forward neural network with ReLU non-
     linearities, regularized with weight decay. We show that the regularized loss
     function is piecewise strongly convex on an  ...