Tag Archives | neural machine translation

luong_cover

The Luong Attention Mechanism

The Luong attention sought to introduce several improvements over the Bahdanau model for neural machine translation, particularly by introducing two new classes of attentional mechanisms: a global approach that attends to all source words, and a local approach that only attends to a selected subset of words in predicting the target sentence.  In this tutorial, […]

Continue Reading 6
bahdanau_cover

The Bahdanau Attention Mechanism

Conventional encoder-decoder architectures for machine translation encoded every source sentence into a fixed-length vector, irrespective of its length, from which the decoder would then generate a translation. This made it difficult for the neural network to cope with long sentences, essentially resulting in a performance bottleneck.  The Bahdanau attention was proposed to address the performance […]

Continue Reading 2