PipeTransformer: Automated Elastic Pipelining for
              Distributed Training of Large-scale Models
            Chaoyang He 1 Shen Li 2 Mahdi Soltanolkotabi 1 Salman Avestimehr 1
             Abstract                Transformer (ViT) (Dosovitskiy et al., 2020) also achieved
                                89% top-1 accuracy in ImageNet, outperforming state-of-
    The size of Transformer models is growing at an     the-art convolutional networks ResNet-152 (He et al., 2016)
    unpreceden ...