publications

2017

1. NeurIPS

   Attention is all you need

   Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin

   In Advances in neural information processing systems, 2017

   Abs Bib

   The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.

   @inproceedings{vaswani2017attention,
     title = {Attention is all you need},
     author = {Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, {\L}ukasz and Polosukhin, Illia},
     booktitle = {Advances in neural information processing systems},
     pages = {5998--6008},
     year = {2017},
   }