OmniNet: Omnidirectional Representations from Transformers

           Yi Tay * 1 Mostafa Dehghani * 2 Vamsi Aribandi 1 3 Jai Gupta 1 Philip Pham 1
                Zhen Qin 1 Dara Bahri 1 Da-Cheng Juan 1 Donald Metzler 1

             Abstract                key defining characteristics in the self-attention mechanism
   This paper proposes Omnidirectional Repre-         is the global receptive field in which each token is accessible
   sentations from Transformers (O MNI N ET). In        to eve ...