Deep Learning for Natural Language Processing
Develop Deep Learning Models for your Natural Language Problems
Deep learning methods are achieving state-of-the-art results on challenging machine learning problems such as describing photos and translating text from one language to another.
In this new laser-focused Ebook written in the friendly Machine Learning Mastery style that you’re used to, finally cut through the math, research papers and patchwork descriptions about natural language processing.
Using clear explanations, standard Python libraries and step-by-step tutorial lessons you will discover what natural language processing is, the promise of deep learning in the field, how to clean and prepare text data for modeling, and how to develop deep learning models for your own natural language processing projects.
About the Ebook:
- PDF format Ebook.
- 8 parts, 30 step-by-step lessons, 414 pages.
- 6 end-to-end tutorial projects.
- 99 Python (.py) code files included.
Clear and Complete Examples.
No Math. Nothing Hidden.
Click to jump straight to the packages.
Working with Text is… important, under-discussed, and HARD
We are awash with text, from books, papers, blogs, tweets, news, and increasingly text from spoken utterances.
Every day, I get questions asking how to develop machine learning models for text data.
Working with text is hard as it requires drawing upon knowledge from diverse domains such as linguistics, machine learning, statistical natural language processing, and these days, deep learning.
The Problem with Text
The problem with modeling text is that it is messy, and machine learning algorithms prefer well defined fixed-length inputs and outputs.
Machine learning algorithms cannot work with raw text directly; the text must be converted into numbers. Specifically, vectors of numbers.
This is called feature extraction or feature encoding and this is one of the key areas where deep learning is really shaking things up.
UNLOCK Natural Language Processing with Deep Learning
Classical linguistic methods for natural language processing required experts in language defining rules to cover specific cases. These worked in narrow cases but turned out to be fragile.
Statistical methods improve upon classical linguistic methods by learning rules and models from data rather than requiring them to be specified in a top-down manner. They result in much better performance, but must still be complemented with hand-crafted augmentations by language experts in order to achieve useful results.
Often, a pipeline of statistical methods are required to achieve a single modeling outcome, such as in the case of machine translation.
Deep learning methods are starting to out-compete the statistical methods on some challenging natural language processing problems with singular and simpler models.
The Promise of Deep Neural Networks for NLP
Deep learning methods are popular, primarily because they are delivering on their promise.
That is not to say that there is no hype around the technology, but that the hype is based on very real results that are being demonstrated across a suite of very challenging artificial intelligence problems from computer vision and natural language processing.
Some of the first large demonstrations of the power of deep learning were in natural language processing, specifically speech recognition. More recently in machine translation.
The 5 promises of deep learning for natural language processing are as follows:
- The Promise of Drop-in Replacement Models. That is, deep learning methods can be dropped into existing natural language systems as replacement models that can achieve commensurate or better performance.
- The Promise of New NLP Models. That is, deep learning methods offer the opportunity of new modeling approaches to challenging natural language problems like sequence-to-sequence prediction.
- The Promise of Feature Learning. That is, that deep learning methods can learn the features from natural language required by the model, rather than requiring that the features be specified and extracted by an expert.
- The Promise of Continued Improvement. That is, that the performance of deep learning in natural language processing is based on real results and that the improvements appear to be continuing and perhaps speeding up.
- The Promise of End-to-End Models. That is, that large end-to-end deep learning models can be fit on natural language problems offering a more general and better-performing approach.
Impressive Applications of Deep Learning
Natural language processing is not “solved“, but deep learning is required to get you to the state-of-the-art on many challenging problems in the field.
Let’s look at 3 examples to give you a snapshot of the results that deep learning is capable of achieving in the field of natural language processing:
1) Automatic Image Caption Generation
Automatic image captioning is the task where, given a photograph, the system must generate a caption that describes the contents of the image.
2) Automatic Translation of Text
Automatic text translation is the task where you are given sentences of text in one language and must translate them into text in another language.
3) Automatic Text Classification
Automatic text classification is the task of assigning a class label given a text document such as a review, tweet, or email.
You can see that developing systems capable of these tasks would be valuable in a wide range of domains and industries.
So, how can you get started and get good at using deep learning for natural language processing fast?
Introducing my new Ebook:
“Deep Learning for Natural Language Processing“
This is the book I wish I had when I was getting started with Deep Learning for NLP.
This book was born out of one thought:
How can I get you proficient with deep learning for NLP as fast as possible?
The Machine Learning Mastery method suggests that the best way of learning this material is by doing. This means the focus of the book is hands-on projects and tutorials. This also means not covering some topics, even topics covered by “everyone else“, like language theory or modeling math.
This book was designed to teach you step-by-step how to bring modern deep learning methods to your natural language processing projects.
You will be led along the critical path from a practitioner interested in natural language processing, to a practitioner that can confidently apply deep learning methods to natural language processing problems.
I want you to get proficient with deep learning for NLP as quickly as you can. I want you using these methods on your project.
Who Is This Book For?
…so is this book right for YOU?
Let’s make sure you are in the right place.
This book is for developers that know some applied machine learning and some deep learning.
Maybe you want or need to start using deep learning for text on your research project or on a project at work. This book was written to help you do that quickly and efficiently by compressing years worth of knowledge and experience into a laser-focused course of hands-on tutorials.
This guide was written in the top-down and results-first style that you’re used to from Machine Learning Mastery.
The lessons in this book assume a few things about you.
You need to know:
- You need to know your way around basic Python and NumPy.
- You need to know your way around basic scikit-learn.
- You need to know your way around basic Keras for deep learning.
You do NOT need to know:
- You do not need to be a math wiz!
- You do not need to be a deep learning expert!
- You do not need to be a master of natural language!
About Your Outcomes
…so what will YOU know after reading it?
This book will teach you how to get results.
After reading and working through this book,
you will know:
- What natural language processing is and why it is challenging.
- What deep learning is and how it is different from other machine learning methods, specifically how it is best understood by deep learning experts.
- The promise of deep learning methods for natural language processing problems as defined by experts in the field.
- How to prepare text data for modeling by hand and using best-of-breed Python libraries such as the natural language toolkit or NLTK.
- How to develop and plot distributed representations of text using word embedding models with the Gensim library.
- How to develop a bag-of-words model, a representation technique that can be used for machine learning and deep learning methods.
- How to develop a neural sentiment analysis model for automatically predicting the class label for a text document.
- How to develop a neural language model, required for any text generating neural network.
- How to develop a photo captioning system to automatically generate textual descriptions of photographs.
- How to develop a neural machine translation system for translating text from one language to another.
This book will NOT teach you how to be a research scientist and all the theory behind why specific methods work. It will teach you how to get results and deliver value on your natural language processing projects.
Exactly What You Need to Know
…30 carefully designed lessons to take you from NLP Beginner to Practitioner
This book was designed to be a practitioners crash course into deep learning for natural language processing.
There are a lot of things you could learn about NLP, from theory to applications to Keras API. My goal is to take you straight to getting results with 30 laser-focused lessons.
I designed the lessons to focus on key skills such as data cleaning, data preparation and modeling required on every single natural language processing project.
Each of the lessons are designed to take you about one hour to read through and complete, excluding the extensions and further reading.
You can choose to work through the lessons one per day, one per week, or at your own pace. I think momentum is critically important, and this book was intended to be read and used, not to sit idle. I would recommend picking a schedule and sticking to it.
The tutorials are divided into eight parts:
- Part 1: Foundations. Discover a gentle introduction to natural language processing, deep learning, and the promise of combining the two, as well as tutorials on how to get started with Keras.
- Part 2: Data Preparation: Discover tutorials that show how to clean, prepare and encode text ready for modeling with neural networks.
- Part 3: Bag-of-Words. Discover the bag-of-words model, a staple representation for machine learning and a good starting point for neural networks for sentiment analysis.
- Part 4: Word Embeddings. Discover a more powerful word representation in word embeddings, how to develop them as standalone models, and how to learn them as part of neural network models.
- Part 5: Text Classification. Discover how to leverage word embeddings and convolutional neural networks to learn spatial invariant models of text for sentiment analysis, a successor to the bag-of-words model.
- Part 6: Language Modeling. Discover how to develop character-based and word-based language models, a technique that is required as part of any modern text generating model.
- Part 7: Image Captioning. Discover how to combine a pre-trained object recognition model with a language model to automatically caption images.
- Part 8: Machine Translation. Discover how to combine two language models to automatically translate text from one language to another.
Below is an overview of the 30 step-by-step tutorial lessons you will complete:
Each lesson was designed to be completed in about 30-to-60 minutes by the average developer.
Part I. Foundations
- Lesson 01: Natural Language Processing
- Lesson 02: Deep Learning
- Lesson 03: Promise of Deep Learning for Natural Language
- Lesson 04: Now to Develop Deep Learning Models With Keras
Part II. Data Preparation
- Lesson 05: How to Clean Text Manually and with NLTK
- Lesson 06: How to Prepare Text Data with scikit-learn
- Lesson 07: How to Prepare Text Data With Keras
Part III. Bag-of-Words
- Lesson 08: The Bag-of-Words Model
- Lesson 09: Prepare Movie Review Data for Sentiment Analysis
- Lesson 10: Neural Bag-of-Words Model for Sentiment Analysis
Part IV. Word Embeddings
- Lesson 11: The Word Embedding Model
- Lesson 12: How to Develop Word Embeddings with Gensim
- Lesson 13: How to Learn and Load Word Embeddings in Keras
Part V. Text Classification
- Lesson 14: Neural Models for Document Classification
- Lesson 15: Develop an Embedding + CNN Model
- Lesson 16: Develop an n-gram CNN Model for Sentiment Analysis
Part VI: Language Modeling
- Lesson 17: Neural Language Modeling
- Lesson 18: Develop a Character-Based Neural Language Model
- Lesson 19: How to Develop a Word-Based Neural Language Model
- Lesson 20: Develop a Neural Language Model for Text Generation
Part VII: Image Captioning
- Lesson 21: Neural Image Caption Generation
- Lesson 22: Neural Network Models for Caption Generation
- Lesson 23: Load and Use a Pre-Trained Object Recognition Model
- Lesson 24: How to Evaluate Generated Text With the BLEU Score
- Lesson 25: How to Prepare a Photo Caption Dataset For Modeling
- Lesson 26: Develop a Neural Image Caption Generation Model
Part VIII: Neural Machine Translation
- Lesson 27: Neural Machine Translation
- Lesson 28: Encoder-Decoder Models for NMT
- Lesson 29: Configure Encoder-Decoder Models for NMT
- Lesson 30: How to Develop a Neural Machine Translation Model
You can see that each part targets a specific learning outcome, and so does each tutorial within each part. This acts as a filter to ensure you are only focused on the things you need to know to get to a specific result and do not get bogged down in the math or near-infinite number of configuration parameters.
The tutorials were not designed to teach you everything there is to know about each of the techniques or natural language processing problems. They were designed to give you an understanding of how they work, how to use them on your projects the fastest way I know how: to learn by doing.
Table of Contents
The screenshot below was taken from the PDF Ebook. It provides you a full overview of the table of contents from the book.
Develop Practical Skills for NLP That You Can Immediately Apply
Discover 4 Different NLP Applications
You will work through 4 different neural natural language processing applications.
- Neural Text Classification. Develop a deep learning model to classify the sentiment of movie reviews as either positive or negative.
- Neural Language Modeling. Develop a neural language model on the text of Plato in order to generate new tracts of text with the same style and flavor as the original.
- Neural Photo Captioning. Develop a model to automatically generate a concise description of ad hoc photographs.
- Neural Machine Translation. Develop a model to translate sentences of text in German to English.
Discover 4 Different Neural Network Models
You will develop 4 different types of neural natural language processing models.
- Neural Bag-of-Words. Develop neural network models that model text as a bag-of-words where word order is ignored.
- Neural Word Embedding. Develop neural network models that model text using a distributed representation.
- Embedding + CNN. Develop deep learning models that combine word embedding representations with convolutional neural networks.
- Encoder-Decoder RNN. Develop recurrent neural networks that use the encoder-decoder architecture.
Take a Sneak Peek Inside The Ebook
Click image to Enlarge.
BONUS: Deep Learning NLP Code Recipes
…you also get 99 fully working Python scripts
Sample Code Recipes
Each recipe presented in the book is standalone, meaning that you can copy and paste it into your project and use it immediately.
- You get one Python script (.py) for each example provided in the book.
This means that you can follow along and compare your answers to a known working implementation of each example in the provided Python files.
This helps a lot to speed up your progress when working through the details of a specific task, such as:
- Cleaning text data.
- Framing a problem.
- Developing a model.
- Evaluating a model.
- Making a prediction.
The provided code was developed in a text editor and intended to be run on the command line. No special IDE or notebooks are required.
All code examples were tested with Python 3 and Keras 2.
All code examples will run on modest and modern computer hardware and were executed on a CPU. No GPUs are required to run the presented examples, although a GPU would make the code run faster.
Python Technical Details
This section provides some technical details about the code provided with the book.
- Python Version: You can use Python 3.
- SciPy: You will use NumPy, Pandas and scikit-learn.
- Keras: You will need Keras version 2 with either a Theano or TensorFlow backend.
- Operating System: You can use Windows, Linux or Mac OS X.
- Hardware: A standard modern workstation will do, no GPUs required.
- Editor: You can use a text editor and run the example from the command line.
Don’t have a Python environment?
The appendix contains step-by-step tutorials showing you exactly how to setup a Python deep learning environment.
About The Author
Hi, I'm Jason Brownlee.
I live in Australia with my wife and son and love to write and code.
I have a computer science background as well as a Masters and Ph.D. degree in Artificial Intelligence.
I’ve written books on algorithms, won and ranked in the top 10% in machine learning competitions, consulted for startups and spent a long time working on systems for forecasting tropical cyclones. (yes I have written tons of code that runs operationally)
I get a lot of satisfaction helping developers get started and get really good at machine learning.
I teach an unconventional top-down and results-first approach to machine learning where we start by working through tutorials and problems, then later wade into theory as we need it.
I'm here to help if you ever have any questions. I want you to be awesome at machine learning.
Download Your Sample Chapter
Do you want to take a closer look at the book? Download a free sample chapter PDF.
Enter your email address and your sample chapter will be sent to your inbox.
Check Out What Customers Are Saying:
You're Not Alone in Choosing Machine Learning Mastery
Trusted by Over 10,000 Practitioners
...including employees from companies like:
...students and faculty from universities like:
and many thousands more...
Absolutely No Risk with...
100% Money Back Guarantee
Plus, as you should expect of any great product on the market, every Machine Learning Mastery Ebook
comes with the surest sign of confidence: my gold-standard 100% money-back guarantee.
100% Money-Back Guarantee
If you're not happy with your purchase of any of the Machine Learning Mastery Ebooks,
just email me within 90 days of buying, and I'll give you your money back ASAP.
No waiting. No questions asked. No risk.
Bring Deep Learning to Your NLP Project TODAY!
Choose Your Package:
You will get the Ebook:
- Deep Learning for NLP
(including bonus source code)
(a great deal!)
All prices are in US Dollars (USD).
(1) Click the button. (2) Enter your details. (3) Download immediately.
Secure Payment Processing With SSL Encryption
Are you a Student, Teacher or Retiree?
Do you have any Questions?
What Are Skills in Machine Learning Worth?
Your boss asks you:
Hey, can you build a predictive model for this?
Imagine you had the skills and confidence to say:
...and follow through.
I have been there. It feels great!
How much is that worth to you?
The industry is demanding skills in machine learning.
The market wants people that can deliver results, not write academic papers.
Business knows what these skills are worth and are paying sky-high starting salaries.
A Data Scientists Salary Begins at:
$100,000 to $150,000.
A Machine Learning Engineers Salary is Even Higher.
What Are Your Alternatives?
You made it this far.
You're ready to take action.
But, what are your alternatives? What options are there?
(1) A Theoretical Textbook for $100+
...it's boring, math-heavy and you'll probably never finish it.
(2) An On-site Boot Camp for $10,000+
...it's full of young kids, you must travel and it can take months.
(3) A Higher Degree for $100,000+
...it's expensive, takes years, and you'll be an academic.
For the Hands-On Skills You Get...
And the Speed of Results You See...
And the Low Price You Pay...
Machine Learning Mastery Ebooks are
And they work. That's why I offer the money-back guarantee.
You're A Professional
The field moves quickly,
...how long can you wait?
You think you have all the time in the world, but...
- New methods are devised and algorithms change.
- New books get released and prices increase.
- New graduates come along and jobs get filled.
Right Now is the Best Time to make your start.
Bottom-up is Slow and Frustrating,
...don't you want a faster way?
Can you really go on another day, week or month...
- Scraping ideas and code from incomplete posts.
- Skimming theory and insight from short videos.
- Parsing Greek letters from academic textbooks.
Targeted Training is your Shortest Path to a result.
Professionals Use Training To Stay On Top Of Their Field
Get The Training You Need!
You don't want to fall behind or miss the opportunity.
Frequently Asked Questions
Why doesn't my payment work?
I am sorry to hear that you're having difficulty.
- Perhaps you can double check that your details are correct, just in case of a typo?
- Perhaps you could try a different payment method, such as PayPal or Credit Card?
- Perhaps you could try my alternative secure payment processor, click here?
- Perhaps you're able to talk to your bank, just in case they blocked the transaction?
If you're still having difficulty, please contact me and I can help investigate further.
Can I get your books for free?
Sorry, I don’t give away free copies of my books.
You can access all of my best free material on my blog.
Can I get a hard copy of your book?
Sorry, I don't have hard copies by design.
The books are written for immediate use, rather than references to sit on the shelf.
My students like to have the PDF open on their screen next to their editor so they can copy-paste code.
Also, the books are updated often to reflect changes to APIs. The field is moving very fast.
I hope that helps explain the rationale.
Are there Kindle or ePub versions of the books?
Sorry, just PDF Ebooks.
This is by design and I put a lot of thought into it. My rationale is as follows:
- I use LaTeX to layout the text and code to give a professional look and I am afraid that EBook readers would mess this up.
- The increase in supported formats would create a maintenance headache that would take a large amount of time away from updating the books and working on new books.
- Most critically, reading on an e-reader or iPad is antithetical to the book-open-next-to-code-editor approach the PDF format was chosen to support.
My materials are playbooks intended to be open on the computer, next to a text editor and a command line. They are not reference texts to be read away from the computer.
Will I get free updates to the books?
All updates are free.
Books are usually updated once every month or two to fix bugs, typos and keep abreast of API changes.
Contact me anytime and check if there have been updates. Let me know what version of the book you have (version is listed on the copyright page).
How do I get access to any bonuses?
After you complete your purchase you will receive an email with a link to download your bundle.
The download will include the book or books and any bonus material.
Is there any digital rights management (DRM)?
Can I print the PDF for my personal use?
In what order should I read your books?
My best advice is to pick a topic that most interests you and start there.
Can I get a customized bundle of books?
Sorry, I cannot create custom bundles of books for you, it would create a maintenance nightmare for me. I’m sure you can understand.
You can see the full catalog of my books and bundles here.
Can I get an evaluation copy of your books?
Sorry, I no longer distribute evaluation copies of my books due to some past abuse of the privilege.
If you are a teacher or lecturer, I’m happy to offer you a student discount.
Contact me and ask for the discount.
Can I get an invoice for my purchase?
Email me with the details of your order (order number or email address used to make the purchase) and details you would like to appear on the invoice (your name, company name and address).
I will create a PDF invoice for you and email it back.
How long do books take to ship?
There are no physical books, therefore no shipping is required.
All books are EBooks that you can download immediately after you complete your purchase.
Do you ship to my country?
There are no physical books, therefore no shipping is required.
All books are EBooks that you can download immediately after you complete your purchase.
I support purchases from any country via PayPal or Credit Card.
Can I have a discount?
I do offer a discount to students, teachers, and retirees.
Note: I only offer discounts on individual books, not on the bundles. This is because the bundles are already heavily discounted.
If you are a student, teacher or a retiree please contact me and ask for the discount.
Do you have any sales, deals, or coupons?
I generally don't do sales.
If I do have a special, such as around the launch of a new book, I only offer it to past customers and subscribers on my email list.
I do offer book bundles that offer a discount for a collection of related books.
Can I get a refund?
I am sorry to hear that you want a refund.
Please contact me directly with your purchase details (order number or email address used to make the purchase) and I will organize a refund.
Will you help me if I have questions?
Please contact me anytime with questions about machine learning or the books.
One question at a time please.
Also, each book has a final chapter on getting more help and further reading and points to resources that you can use to get more help.
Do I need to be a good programmer?
Not at all.
My material requires that you have a programmers mindset of thinking in procedures and learning by doing.
You do not need to be an excellent programmer to read and learn about machine learning algorithms.
How much math do I need to know?
No background in statistics, probability or linear algebra is required.
I teach using a top-down and results-first approach to machine learning. You will learn by doing, not learn by theory.
There are no derivations.
Any questions presented are explained in full and are only provided to make the explanation clearer, not more confusing.
How much machine learning do I need to know?
Only a little.
If you are a reader of my blog posts, then you know enough to get started.
I do my best to lead you through what you need to know, step-by-step.
How long will the book take me to complete?
I recommend reading one chapter per day.
Some students finish the book in a weekend.
Most students finish the book in a few weeks by working through it during nights and weekends.
How are your books different to other books?
My books are playbooks. Not textbooks.
They have no deep explanations of theory, just working examples that are laser-focused on the information that you need to know to bring machine learning to your project.
My books are not for everyone, they are carefully designed for practitioners that need to get results, fast.
How are your books different from the blog?
The books are a concentrated and more convenient version of what I put on the blog.
I design my books to be a combination of lessons and projects to teach you how to use a specific machine learning tool or library and then apply it to real predictive modeling problems.
The books get updated with bug fixes, updates for API changes and the addition of new chapters, and these updates are totally free.
I do put some of the book chapters on the blog as examples, but they are not tied to the surrounding chapters or the narrative that a book offers and do not offer the standalone code files.
With each book, you also get all of the source code files used in the book that you can use as recipes to jump-start your own predictive modeling problems.
How are the 2 algorithms books different?
The book “Master Machine Learning Algorithms” is for programmers and non-programmers alike that learn through worked examples. It teaches you how 10 top machine learning algorithms work, with worked examples in arithmetic, not code (and spreadsheets) that show how each model learns and makes predictions.
The book “Machine Learning Algorithms From Scratch” is for programmers that learn by writing code to understand. It provides step-by-step tutorials on how to implement top algorithms as well as how to load data, evaluate models and more. It has less on how the algorithms work, instead focusing exclusively on how to implement each in code.
The two books can support each other.
Is there a team or company-wide license?
Due to abuse of the privilege, I only support purchases by individuals.
Is there a license for libraries?
Sorry, I only support purchases by individuals.
Do you have videos?
I only have tutorial lessons and projects in text format.
This is by design. I used to have video content and I found the completion rate much lower.
I want you to put the material into practice. I have found that text-based tutorials are the best way of achieving this.
After reading and working through the tutorials you are far more likely to apply what you have learned.
What operating systems are supported?
Linux, Mac OS X and Windows.
Can you be my mentor or coach?
Thanks for asking. I would love to help, but I just don't have the capacity.
I try to help as many people as possible through my blog and books.
Can I purchase from Amazon (or elsewhere)?
My books can only be purchased from my website.
The reason is that I am a small business and I want a direct relationship with you, my customer, so that I can offer personal support and send out updates about your book and new stuff I am working on.
I hope you can understand my rationale.
What if my download link expires?
It is possible that your link to download your purchase will expire after a few days.
This is a security precaution.
Please contact me and I will resend you purchase receipt with an updated download link.
Can I use your code in my own project?
But, understand that all code was developed and provided for educational purposes only and that I take no responsibility for it, what it might do or how you might use it.
Do you have another question?