Learning Under Delayed Feedback: Implicitly Adapting to Gradient Delays

              Rotem Zamir Aviv 1 Ido Hakimi 2 Assaf Schuster 2 Kfir Y. Levy 1 3

             Abstract                 ceed. Such methods are easier to analyze, yet their per-
    We consider stochastic convex optimization          formance depends on the slowest machine and they require
    problems, where several machines act asyn-          large communication overheads. Conversely, in the asyn-
    chronously in parallel ...