Which Transformer architecture fits my data?
               A vocabulary bottleneck in self-attention

               Noam Wies 1 Yoav Levine 1 Daniel Jannai 1 Amnon Shashua 1

              Abstract              unchanged, the chosen ratio between the number of self-
    After their successful debut in natural language    attention layers (depth) and the dimension of the internal
    processing, Transformer architectures are now be-    representation (width) varies greatly across different appl ...