Search results for "beam search"

How to Implement Beam Search Decoder for Natural Language Processing

How to Implement a Beam Search Decoder for Natural Language Processing

Natural language processing tasks, such as caption generation and machine translation, involve generating sequences of words. Models developed for these problems often operate by generating probability distributions across the vocabulary of output words and it is up to decoding algorithms to sample the probability distributions to generate the most likely sequences of words. In this […]

Continue Reading 41
How to Configure an Encoder-Decoder Model for Neural Machine Translation

How to Configure an Encoder-Decoder Model for Neural Machine Translation

The encoder-decoder architecture for recurrent neural networks is achieving state-of-the-art results on standard machine translation benchmarks and is being used in the heart of industrial translation services. The model is simple, but given the large amount of data required to train it, tuning the myriad of design decisions in the model in order get top […]

Continue Reading 16
Encoder-Decoder Recurrent Neural Network Models for Neural Machine Translation

Encoder-Decoder Recurrent Neural Network Models for Neural Machine Translation

The encoder-decoder architecture for recurrent neural networks is the standard neural machine translation method that rivals and in some cases outperforms classical statistical machine translation methods. This architecture is very new, having only been pioneered in 2014, although, has been adopted as the core technology inside Google’s translate service. In this post, you will discover […]

Continue Reading 16
What is Teacher Forcing for Recurrent Neural Networks?

What is Teacher Forcing for Recurrent Neural Networks?

Teacher forcing is a method for quickly and efficiently training recurrent neural network models that use the ground truth from a prior time step as input. It is a network training method critical to the development of deep learning language models used in machine translation, text summarization, and image captioning, among many other applications. In […]

Continue Reading 34
Encoder-Decoder Deep Learning Models for Text Summarization

Encoder-Decoder Deep Learning Models for Text Summarization

Text summarization is the task of creating short, accurate, and fluent summaries from larger text documents. Recently deep learning methods have proven effective at the abstractive approach to text summarization. In this post, you will discover three different models that build on top of the effective Encoder-Decoder architecture developed for sequence-to-sequence prediction in machine translation. […]

Continue Reading 6
Example of annotation regions of an image with descriptions

How to Automatically Generate Textual Descriptions for Photographs with Deep Learning

Captioning an image involves generating a human readable textual description given an image, such as a photograph. It is an easy problem for a human, but very challenging for a machine as it involves both understanding the content of an image and how to translate this understanding into natural language. Recently, deep learning methods have […]

Continue Reading 26
Implementation Patterns for the Encoder-Decoder RNN Architecture with Attention

Implementation Patterns for the Encoder-Decoder RNN Architecture with Attention

The encoder-decoder architecture for recurrent neural networks is proving to be powerful on a host of sequence-to-sequence prediction problems in the field of natural language processing. Attention is a mechanism that addresses a limitation of the encoder-decoder architecture on long sequences, and that in general speeds up the learning and lifts the skill of the […]

Continue Reading 6