Archive | Deep Learning for Natural Language Processing

How to Develop a Character-Based Neural Language Model in Keras

How to Develop a Character-Based Neural Language Model in Keras

A language model predicts the next word in the sequence based on the specific words that have come before it in the sequence. It is also possible to develop language models at the character level using neural networks. The benefit of character-based language models is their small vocabulary and flexibility in handling any words, punctuation, […]

Continue Reading 51
How to Develop Word-Based Neural Language Models in Python with Keras

How to Develop Word-Based Neural Language Models in Python with Keras

Language modeling involves predicting the next word in a sequence given the sequence of words already present. A language model is a key element in many natural language processing models such as machine translation and speech recognition. The choice of how the language model is framed must match how the language model is intended to […]

Continue Reading 75
Gentle Introduction to Statistical Language Modeling and Neural Language Models

Gentle Introduction to Statistical Language Modeling and Neural Language Models

Language modeling is central to many important natural language processing tasks. Recently, neural-network-based language models have demonstrated better performance than classical methods both standalone and as part of more challenging natural language processing tasks. In this post, you will discover language modeling for natural language processing. After reading this post, you will know: Why language […]

Continue Reading 8
How to Develop a Word Embedding Model for Predicting Movie Review Sentiment

How to Develop a Deep Convolutional Neural Network for Sentiment Analysis (Text Classification)

Develop a Deep Learning Model to Automatically Classify Movie Reviews as Positive or Negative in Python with Keras, Step-by-Step. Word embeddings are a technique for representing text where different words with similar meaning have a similar real-valued vector representation. They are a key breakthrough that has led to great performance of neural network models on […]

Continue Reading 142
How to Define an Encoder-Decoder Sequence-to-Sequence Model for Neural Machine Translation in Keras

How to Develop a Seq2Seq Model for Neural Machine Translation in Keras

The encoder-decoder model provides a pattern for using recurrent neural networks to address challenging sequence-to-sequence prediction problems, such as machine translation. Encoder-decoder models can be developed in the Keras Python deep learning library and an example of a neural machine translation system developed with this model has been described on the Keras blog, with sample […]

Continue Reading 198
Best Practices for Document Classification with Deep Learning

Best Practices for Text Classification with Deep Learning

Text classification describes a general class of problems such as predicting the sentiment of tweets and movie reviews, as well as classifying email as spam or not. Deep learning methods are proving very good at text classification, achieving state-of-the-art results on a suite of standard academic benchmark problems. In this post, you will discover some […]

Continue Reading 46
How to Develop a Deep Learning Bag-of-Words Model for Predicting Sentiment in Movie Reviews

How to Develop a Deep Learning Bag-of-Words Model for Sentiment Analysis (Text Classification)

Movie reviews can be classified as either favorable or not. The evaluation of movie review text is a classification problem often called sentiment analysis. A popular technique for developing sentiment analysis models is to use a bag-of-words model that transforms documents into vectors where each word in the document is assigned a score. In this […]

Continue Reading 96
How to Prepare Movie Review Data for Sentiment Analysis

How to Prepare Movie Review Data for Sentiment Analysis (Text Classification)

Text data preparation is different for each problem. Preparation starts with simple steps, like loading data, but quickly gets difficult with cleaning tasks that are very specific to the data you are working with. You need help as to where to begin and what order to work through the steps from raw data to data […]

Continue Reading 45
Feeding Hidden State as Input to Decoder

How Does Attention Work in Encoder-Decoder Recurrent Neural Networks

Attention is a mechanism that was developed to improve the performance of the Encoder-Decoder RNN on machine translation. In this tutorial, you will discover the attention mechanism for the Encoder-Decoder model. After completing this tutorial, you will know: About the Encoder-Decoder model and attention mechanism for machine translation. How to implement the attention mechanism step-by-step. […]

Continue Reading 32