Ensemble Learning Algorithms With Python
Make Better Predictions with Bagging, Boosting, and Stacking
Predictive performance is the most important concern on many classification and regression problems. Ensemble learning algorithms combine the predictions from multiple models and are designed to perform better than any contributing ensemble member.
Cut through the equations, Greek letters, and confusion, and discover the specialized ensemble learning algorithms that you need to know to get the most out of your data on your next project.
Using clear explanations, standard Python libraries, and step-by-step tutorial lessons, you will discover how to confidently and effectively improve predictive modeling performance using ensemble algorithms.
About this Ebook:
- Read on all devices: English PDF format EBook, no DRM.
- Tons of tutorials: 30 step-by-step lessons, 450 pages.
- Foundations: intuitions ensemble learning, diversity, more.
- Many algorithms: xgboost, extra trees, blending, more.
- Working code: 186 Python (.py) code files included.
Clear, Complete End-to-End Examples.
Convinced?
…so What is Ensemble Learning?
Ensemble learning algorithms combine the predictions of two or more models.
The idea of ensemble learning is closely related to the idea of the “wisdom of crowds“. This is where many different independent decisions, choices or estimates are combined into a final outcome that is often more accurate than any single contribution.
This is the core idea behind major aspects of modern society, such as a scientific peer review, a jury of peers, and seeking a second opinion. It is an alternative to seeking out and taking the advice of an expert.
In applied machine learning, it means combining the predictions from multiple models trained on your dataset, instead of seeking the single best performing model.
…so Why Use an Ensemble Learning Model?
An ensemble learning model is often more complex than using a single machine learning model.
This may require a more sophisticated technique for preparing the model, more computational resources to train the model, and less introspection available to understand why a specific prediction was made.
Nevertheless, there are 2 main reasons to use ensemble learning algorithms, they are:
Better Robustness
Many machine learning algorithms give different predictions each time a model is trained on the same data or even slightly different data.
This is referred to as the variance in the predictions or the stability of the model.
Ensemble learning algorithms can give more stable predictions than a single model.
Better Predictions
There are some problem domains and predictive modeling tasks where better predictive performance is a more important consideration than anything else.
This is particularly the case in competitive machine learning where predictive skill is the only outcome that is considered.
Ensemble learning algorithms can give better predictive skill than using a single model.
Ensembles are commonly used to stabilize the predictions made by deep learning models given both the stochastic nature of the model architecture that is initialized with random weights and the stochastic gradient descent learning algorithm.
The best case for the benefit for ensemble learning are machine learning competitions, such as the One Million Dollar Netflix prize where the winning team used an ensemble of hundreds of models, and Kaggle competitions where top participants and winners almost universally use ensemble learning methods.
Ensemble Learning Used to be REALLY Hard!
Ensemble learning used to be more art than science.
Before the development of standard techniques, you had to design your own ensemble learning algorithms.
This was hard because it required carefully managing the diversity of ensemble members and carefully selecting how predictions are combined.
Ensemble diversity refers to the differences in the predictions or prediction errors made by models that contribute to the ensemble. Ideally, each ensemble member would be statistically independent, meaning it would make good predictions in a completely different way to all other ensemble members. This is not possible as we only have a single training dataset.
Combination methods refers to the specific way that the output from each ensemble member is used to make a final prediction. This could involve using a simple statistical method as the arthritic mean or voting, although could involve weighing predictions by the skill of the contributing model or even using another model to learn how best combine predictions.
Thankfully, a suite of standard ensemble learning algorithms have been developed.
…we now have:
Standard Ensemble Learning Algorithms
The 1990s and early 2000s were the boom time for the development of ensemble learning algorithms.
This was the time that the major types of ensemble algorithms were developed, extended, and improved upon.
This gave us three main families of ensemble learning algorithms to choose from, including:
Bagging Ensembles
These ensembles involve training many high variance models on different samples of the training dataset.
Some algorithms include:
- Bagged Decision Trees
- Random Subspaces
- Random Forest
- Extra Trees
- Custom Bagging
Boosting Ensembles
These ensembles involve adding models sequentially to correct the predictions of prior models.
Some algorithms include:
- AdaBoost
- Gradient Boosting Machine
- Stochastic Gradient Boosting
- XGBoost
- LightGBM
Stacking Ensembles
These ensembles involve using a model to learn how to best combine the predictions from models.
Some algorithms include:
- Voting
- Weighted Average
- Blending
- Stacking
- Super Learner
In some cases, algorithms were developed by careful and systematic trial and error, in other cases algorithms were inspired by an analogy or metaphor, and some were based on heavy theoretical machine learning.
The large number of methods and complexity of each method made it very challenging to use these proven ensemble learning algorithms in practice.
In many cases, it is very easy to make a small implementation error which results in data leakage and overly optimistic estimates of model performance. In other cases, the algorithms are so slow to train, that they cannot be realistically sized training dataset.
Thankfully, modern machine learning libraries like the scikit-learn library in Python provide easy-to-use implementations of the sophisticated algorithms and tools to avoid data leakage. Third party libraries like XGBoost and LightGBM provide highly optimized implementations designed for speed of execution, allowing some of the computationally more expensive methods to be scaled to much larger datasets.
It is a great time to bring ensemble learning to your predictive modeling projects!
Introducing My New EBook:
“Ensemble Learning Algorithms With Python“
Welcome to the EBook: Ensemble Learning Algorithms With Python.
I designed this book to teach machine learning practitioners, like you, step-by-step how to configure and use the most powerful ensemble learning techniques with examples in Python.
This book was carefully designed to help you bring a wide variety of the proven and custom ensemble learning techniques to your next project.
The tutorials were designed to teach you these techniques the fastest and most effective way that I know how: to learn by doing. With executable code that you can run to develop the intuitions required, and that you can copy-and-paste into your project and immediately get a result.
Ensemble learning is important to machine learning, and I believe that if it is taught at the right level for practitioners, it can be a fascinating, fun, directly applicable, and immeasurably useful toolbox of techniques.
I hope that you agree.
Convinced?
Click to jump straight to the packages.
…so, is this book right for YOU?
Who Is This Book For?
Let’s make sure you are in the right place.
This book is for developers that may know some applied machine learning. Maybe you know how to work through a predictive modeling problem end-to-end, or at least most of the main steps, with popular tools.
This guide was written in the top-down and results-first machine learning style that you’re used to from Machine Learning Mastery.
The lessons in this book do assume a few things about you, such as:
You need to know:
- You know your way around basic Python for programming.
- You may know some basic NumPy for array manipulation.
- You may know some basic scikit-learn for modeling.
You do NOT need to be:
- You do not need to be a math wiz!
- You do not need to be a master programmer!
- You do not need to be a machine learning expert!
About Your Outcomes
…so what will YOU know after reading this book?
After reading and working through this book, you will know:
- The intuition behind drawing upon a crowd or multiple experts when making important decisions and how this intuition carries over to ensemble learning algorithms.
- The benefits of ensemble learning techniques for predictive modeling for both lifting predictive skill and improving model robustness.
- How to develop and evaluate multi-model algorithms for classification and regression problems, providing a precursor to ensemble learning.
- How to develop, configure, and evaluate bagging ensembles for classification and regression predictive modeling problems.
- How to develop and evaluate extensions to bagging, such as random subspace, random forest, and extra trees ensembles.
- How to develop, configure, and evaluate adaptive boosting (AdaBoost) and gradient ensembles for classification and regression predictive modeling problems.
- How to develop and evaluate efficient implementations of gradient boosting ensembles, such as extreme gradient boosting (XGBoost) and light gradient boosting machines (LightGBM).
- How to develop, configure, and evaluate stacking ensembles for classification and regression predictive modeling problems.
- How to develop and evaluate simpler stacking ensembles such as voting and weighted average ensembles.
- How to develop and evaluate extensions to stacking, such as model blending and super learner ensembles.
But, what if…?
Do you have some doubts? Let me see if I can help.
What if I Am New to Machine Learning?
This book does not assume you have a background in machine learning.
That being said, I do recommend that you learn how to work through a predictive modeling problem first. It will give you the context for the challenge of ensemble learning.
What if I Am Just a Developer?
Perfect. I wrote this book for you.
What if My Math is Really Poor?
Perfect. This book is for you.
No math is required, other than some basic arithmetic.
What if I Am Not a Python Programmer?
You can handle this book if you are a programmer in another language, even if you are not experienced in Python.
Everything is demonstrated with a small code example that you can run directly.
All code is provided for you to play with, modify, and learn from.
The book even has an appendix to show you how to set up Python on your workstation.
What if I Am Working Through another Machine Learning Course?
Excellent!
This book is not a substitute for an undergraduate course in machine learning or a textbook for such a course, although it is a great complement to such materials.
What Exactly Is In This Book?
…see the table of contents
This book was designed around major ensemble learning techniques that are directly relevant to real-world problems.
There are a lot of things you could learn about ensemble learning, from theory to abstract concepts to APIs. My goal is to take you straight to developing an intuition for the elements you must understand with laser-focused tutorials.
The tutorials were designed to teach you how to get results with ensemble learning methods. As such, the tutorials give you the tools to both rapidly understand and apply each technique or operation. There is a mixture of both tutorial lessons and projects to both introduce the methods and give plenty of examples and opportunities to practice using them.
Each of the tutorials is designed to take you about one hour to read through and complete, excluding the extensions and further reading.
You can choose to work through the lessons one per day, one per week, or at your own pace. I think momentum is critically important, and this book is intended to be read and used, not to sit idle.
I would recommend picking a schedule and sticking to it.
The tutorials are divided into six parts; they are:
- Part 1: Foundation: Discover the power of ensemble learning techniques, why they are important to getting good performance on your project and how to develop an intuition for what is being learned.
- Part 2: Background: Discover the background required for ensemble learning including the diversity of ensemble members, techniques for combining predictions, the complexity of ensemble models, and the main types of ensemble methods.
- Part 3: Multiple Models: Discover machine learning techniques that involve explicitly using multiple models which provide the foundation for ensemble learning methods.
- Part 4: Bagging: Discover bootstrap aggregation known as the bagging family of ensemble learning techniques including random forest, extra trees and related methods.
- Part 5: Boosting: Discover the boosting family of ensemble learning techniques including adaptive boosting, gradient boosting and modern efficient implementations like extreme gradient boosting and light gradient boosting machines.
- Part 6: Stacking: Discover the stacked generalization or stacking family of ensemble learning methods including voting, blending, and related methods.
Lessons Overview
Below is an overview of the 30 step-by-step tutorial lessons you will work through:
Each lesson was designed to be completed in about 30-to-60 minutes by an average developer.
Part 1: Foundation
- Lesson 01: What is Ensemble Learning
- Lesson 02: Why Use Ensemble Learning
- Lesson 03: Intuition for Ensemble Learning
Part 2: Background
- Lesson 04: Ensemble Member Diversity
- Lesson 05: Techniques for Combining Predictions
- Lesson 06: Ensemble Complexity
- Lesson 07: Types of Ensemble Algorithms
Part 3: Multiple Models
- Lesson 08: One-vs-Rest and One-vs-All Models
- Lesson 09: Error-Correcting Output Codes
- Lesson 10: Multiple Regression Models
- Lesson 11: Dynamic Classifier Selection
- Lesson 12: Dynamic Ensemble Selection
- Lesson 13: Mixture of Experts Ensemble
Part 4: Bagging
- Lesson 14: Bagged Decision Trees Ensemble
- Lesson 15: Random Subspace Ensemble
- Lesson 16: Feature Selection Bagging Ensemble
- Lesson 17: Random Forest Ensemble
- Lesson 18: Extra Trees Ensemble
- Lesson 19: Data Preparation Bagging Ensemble
Part 5: Boosting
- Lesson 20: Strong vs Weak Learners
- Lesson 21: Adaptive Boost Ensemble
- Lesson 22: Gradient Boosting Ensemble
- Lesson 23: Extreme Gradient Boosting Ensemble
- Lesson 24: Light Gradient Boosting Machine Ensemble
Part 6: Stacking
- Lesson 25: Voting Ensemble
- Lesson 26: Weighted Average Ensemble
- Lesson 27: Ensemble Member Selection
- Lesson 28: Stacking Ensemble
- Lesson 29: Blending Ensemble
- Lesson 30: Super Learner Ensemble
Appendix
- Appendix A: Getting help
- Appendix B: How to Setup a Workstation for Python
You can see that each part targets a specific learning outcome, and so does each tutorial within each part. This acts as a filter to ensure you are only focused on the things you need to know to get to a specific result and do not get bogged down in the math or near-infinite number of digressions.
The tutorials were not designed to teach you everything there is to know about each of the theories or techniques. They were designed to give you an understanding of how they work, how to use them, and how to interpret the results the fastest way I know how: to learn by doing.
EBook Table of Contents
The screenshot below was taken from the PDF Ebook. It provides you a full overview of the table of contents from the book.
Ensemble Learning Algorithms With Python Table of Contents
Here’s Everything You’ll Get…
in “Ensemble Learning Algorithms With Python“
A digital download that contains everything you need, including:
- Clear descriptions to help you understand ensemble learning algorithms for applied machine learning.
- Step-by-step Python tutorials to show you exactly how to apply each technique and algorithm.
- End-to-end self-contained examples that give you everything you need in each tutorial without assuming prior knowledge.
- Python source code recipes for every example in the book so that you can run the tutorial code in seconds.
- Digital Ebook in PDF format so that you can have the book open side-by-side with the code and see exactly how each example works.
Resources you need to go deeper, when you need to, including:
- The best sources of information on the Python ecosystem including the SciPy, NumPy, and scikit-learn libraries.
- The best places online where you can ask your challenging questions and actually get a response.
- The best books, and articles to learn more about each technique covered.
Foundations required for developing and using ensemble learning algorithms, including:
- The importance of statistical independence between ensemble members.
- The consideration of correlation between the predictions and prediction errors made by ensemble members.
- The diversity of ensemble members and techniques designed to control or increase ensemble diversity.
- The requirement of a combination method to pool the output of multiple models.
- Techniques for combining class labels and class membership probabilities for classification tasks.
- Techniques for combining numerical values for regression predictive modeling.
- How more complex ensemble learning models can be reconciled with Occam’s razor.
- The three types of ensemble learning algorithms to focus on.
Stacking algorithms that learn how to best combine the predictions from multiple models, including:
- The essence of stacked generalization ensemble learning for classification and regression.
- How to develop voting ensembles including hard voting and soft voting for classification tasks.
- How to develop weighted average and weighted sum ensemble algorithms.
- How to grow an ensemble from scratch or prune ensemble members from a model.
- How to develop, configure, and evaluate stacking ensembles that learn how to combine predictions.
- How to develop simpler blending ensembles common in large machine learning competitions.
- How to develop more complex super learner ensembles.
Background on the field of ensemble learning to give you the context you need, including:
- The wisdom of crowds principle that provides the basis for ensemble learning.
- How to define an ensemble learning algorithm.
- The two benefits of using ensemble learning for predictive modeling.
- The intuition for ensemble learning on classification and regression tasks.
Multiple model techniques that provide the foundation for ensemble learning, including:
- The essence of multi-model machine learning techniques for classification and regression.
- How to develop One-vs-Rest (OvR) and One-vs-One (OvO) multi-model algorithms for multi-class classification.
- How to develop output error correcting code multi-model algorithms for multi-class classification.
- How to develop multi-model algorithms for multi-output regression tasks.
- Dynamic classifier selection that fits many models on a dataset and choose the best one for each prediction.
- Dynamic ensemble selection that selects the best subset of models for each prediction made.
- Mixture of experts that uses a divide and conquer approach to ensemble learning.
Bagging algorithms that combine models fit on different training data, including:
- The essence of bootstrap aggregation ensemble learning for classification and regression.
- How to develop, configure, and evaluate bagged decision tree ensembles.
- How to develop random subspace ensembles that train members on different subsets of input features.
- How to explore custom feature selection based bagging ensembles.
- How to develop random forest ensembles that combine ideas from bagging and random subspace ensembles.
- How to develop extra trees ensembles that relax the constraints of random forest.
- How to explore custom data preparation based bagging ensembles.
Boosting algorithms that sequentially add models to correct prior models, including:
- The essence of boosting based ensemble learning for classification and regression.
- The difference between strong learners and weak learners and their part in boosting ensembles.
- How to develop, configure, and evaluate gradient boosting ensembles.
- How to reduce the computational cost of training a gradient boosting model with XGBoost.
- How to improve performance and reduce the complexity of gradient boosting models with LightGBM.
What More Do You Need?
Take a Sneak Peek Inside The EBook
Click an image to Enlarge.
BONUS: Ensemble Learning Algorithms With Python Code Recipes
….you also get 186 fully working Python scripts
Sample Code Recipes
Each recipe presented in the book is standalone, meaning that you can copy and paste it into your project and use it immediately.
- You get one Python script (.py) for each example provided in the book.
This means that you can follow along and compare your answers to a known working implementation of each example in the provided Python files.
This helps a lot to speed up your progress when working through the details of a specific task, such as:
- Multiple models for classification.
- Multiple models for regression.
- Tuning hyperparameters for ensembles.
- Creating customized ensembles.
- Configuring gradient boosting.
- Using third-party libraries for gradient boosting.
- Comparing the performance of different models.
- Implementing blending from scratch.
- Avoiding data leakage with pipelines.
- Coding a super learner from scratch.
The provided code was developed in a text editor and is intended to be run on the command line. No special IDE or notebooks are required.
All code examples were designed and tested with Python 3.6+.
All code examples will run on modest and modern computer hardware and were executed on a CPU.
Python Technical Details
This section provides some technical details about the code provided with the book.
- Python Version: You can use Python 3.6 or higher.
- SciPy: You will use NumPy, SciPy, and scikit-learn APIs.
- OS: You can use Windows, Linux, or MacOS.
- Hardware: A standard modern workstation will do.
- Editor: You can use a text editor and run the example from the command line.
Don’t have a Python environment?
No problem!
The appendix contains a step-by-step tutorial showing you exactly how to set up a Python machine learning environment.
Ensemble Learning Algorithms With Python Code Recipes
About The Author
Hi, I'm Jason Brownlee. I run this site and I wrote and published this book.
I live in Australia with my wife and sons. I love to read books, write tutorials, and develop systems.
I have a computer science and software engineering background as well as Masters and PhD degrees in Artificial Intelligence with a focus on stochastic optimization.
I've written books on algorithms, won and ranked well in competitions, consulted for startups, and spent years in industry. (Yes, I have spend a long time building and maintaining REAL operational systems!)
I get a lot of satisfaction helping developers get started and get really good at applied machine learning.
I teach an unconventional top-down and results-first approach to machine learning where we start by working through tutorials and problems, then later wade into theory as we need it.
I'm here to help if you ever have any questions. I want you to be awesome at machine learning.
Download Your Sample Chapter
Do you want to take a closer look at the book? Download a free sample chapter PDF.
Enter your email address and your sample chapter will be sent to your inbox.
Click here to Download Your Sample Chapter
You're Not Alone in Choosing Machine Learning Mastery
Trusted by Over 53,938 Practitioners
...including employees from companies like:
...students and faculty from universities like:
and many thousands more...
Absolutely No Risk with...
100% Money Back Guarantee
Plus, as you should expect of any great product on the market, every Machine Learning Mastery Ebook
comes with the surest sign of confidence: my gold-standard 100% money-back guarantee.
100% Money-Back Guarantee
If you're not happy with your purchase of any of the Machine Learning Mastery Ebooks,
just email me within 90 days of buying, and I'll give you your money back ASAP.
No waiting. No questions asked. No risk.
…it’s time to take the next step.
Bring Modern Ensemble Learning Techniques to
Your Predictive Modeling Projects NOW!
Choose Your Package:
Basic Package
You will get the Ebook:
- Ensemble Learning Algorithms With Python
(including bonus source code)
(a great deal!)
All prices are in US Dollars (USD).
(1) Click the button. (2) Enter your details. (3) Download immediately.
Secure Payment Processing With SSL Encryption
Do you have any Questions?
What Are Skills in Machine Learning Worth?
Your boss asks you:
Hey, can you build a predictive model for this?
Imagine you had the skills and confidence to say:
"YES!"
...and follow through.
I have been there. It feels great!
How much is that worth to you?
The industry is demanding skills in machine learning.
The market wants people that can deliver results, not write academic papers.
Business knows what these skills are worth and are paying sky-high starting salaries.
A Data Scientists Salary Begins at:
$100,000 to $150,000.
A Machine Learning Engineers Salary is Even Higher.
What Are Your Alternatives?
You made it this far.
You're ready to take action.
But, what are your alternatives? What options are there?
(1) A Theoretical Textbook for $100+
...it's boring, math-heavy and you'll probably never finish it.
(2) An On-site Boot Camp for $10,000+
...it's full of young kids, you must travel and it can take months.
(3) A Higher Degree for $100,000+
...it's expensive, takes years, and you'll be an academic.
OR...
For the Hands-On Skills You Get...
And the Speed of Results You See...
And the Low Price You Pay...
Machine Learning Mastery Ebooks are
Amazing Value!
And they work. That's why I offer the money-back guarantee.
You're A Professional
The field moves quickly,
...how long can you wait?
You think you have all the time in the world, but...
- New methods are devised and algorithms change.
- New books get released and prices increase.
- New graduates come along and jobs get filled.
Right Now is the Best Time to make your start.
Bottom-up is Slow and Frustrating,
...don't you want a faster way?
Can you really go on another day, week or month...
- Scraping ideas and code from incomplete posts.
- Skimming theory and insight from short videos.
- Parsing Greek letters from academic textbooks.
Targeted Training is your Shortest Path to a result.
Professionals Stay On Top Of Their Field
Get The Training You Need!
You don't want to fall behind or miss the opportunity.
Frequently Asked Questions
Customer Questions (78)
Thanks for your interest.
Sorry, I do not support third-party resellers for my books (e.g. reselling in other bookstores).
My books are self-published and I think of my website as a small boutique, specialized for developers that are deeply interested in applied machine learning.
As such I prefer to keep control over the sales and marketing for my books.
I’m sorry, I don’t support exchanging books within a bundle.
The collections of books in the offered bundles are fixed.
My e-commerce system is not sophisticated and it does not support ad-hoc bundles. I’m sure you can understand. You can see the full catalog of books and bundles here:
If you have already purchased a bundle and would like to exchange one of the books in the bundle, then I’m very sorry, I don’t support book exchanges or partial refunds.
If you are unhappy, please contact me directly and I can organize a refund.
Thanks for your interest.
I’m sorry, I cannot create a customized bundle of books for you. It would create a maintenance nightmare for me. I’m sure you can understand.
My e-commerce system is not very sophisticated. It cannot support ad-hoc bundles of books or the a la carte ordering of books.
I do have existing bundles of books that I think go well together.
You can see the full catalog of my books and bundles available here:
Sorry, I don’t sell hard copies of my books.
All of the books and bundles are Ebooks in PDF file format.
This is intentional and I put a lot of thought into the decision:
- The books are full of tutorials that must be completed on the computer.
- The books assume that you are working through the tutorials, not reading passively.
- The books are intended to be read on the computer screen, next to a code editor.
- The books are playbooks, they are not intended to be used as references texts and sit the shelf.
- The books are updated frequently, to keep pace with changes to the field and APIs.
I hope that explains my rationale.
If you really do want a hard copy, you can purchase the book or bundle and create a printed version for your own personal use. There is no digital rights management (DRM) on the PDF files to prevent you from printing them.
Sorry, I cannot create a purchase order for you or fill out your procurement documentation.
You can complete your purchase using the self-service shopping cart with Credit Card or PayPal for payment.
After you complete the purchase, I can prepare a PDF invoice for you for tax or other purposes.
Sorry, no.
I cannot issue a partial refund. It is not supported by my e-commerce system.
If you are truly unhappy with your purchase, please contact me about getting a full refund.
I stand behind my books, I know the tutorials work and have helped tens of thousands of readers.
I am sorry to hear that you want a refund.
Please contact me directly with your purchase details:
- Book Name: The name of the book or bundle that you purchased.
- Your Email: The email address that you used to make the purchase (note, this may be different to the email address you used to pay with via PayPal).
- Order Number: The order number in your purchase receipt email.
I will then organize a refund for you.
I would love to hear why the book is a bad fit for you.
Anything that you can tell me to help improve my materials will be greatly appreciated.
I have a thick skin, so please be honest.
Sample chapters are provided for each book.
Each book has its own webpage, you can access them from the catalog.
On each book’s page, you can access the sample chapter.
- Find the section on the book’s page titled “Download Your Sample Chapter“.
- Click the link, provide your email address and submit the form.
- Check your email, you will be sent a link to download the sample.
If you have trouble with this process or cannot find the email, contact me and I will send the PDF to you directly.
Yes.
I can provide an invoice that you can use for reimbursement from your company or for tax purposes.
Please contact me directly with your purchase details:
- The name of the book or bundle that you purchased.
- The email address that you used to make the purchase.
- Ideally, the order number in your purchase receipt email.
- Your full name/company name/company address that you would like to appear on the invoice.
I will create a PDF invoice for you and email it back.
Sorry, I no longer distribute evaluation copies of my books due to some past abuse of the privilege.
If you are a teacher or lecturer, I’m happy to offer you a student discount.
Contact me directly and I can organize a discount for you.
Sorry, I do not offer Kindle (mobi) or ePub versions of the books.
The books are only available in PDF file format.
This is by design and I put a lot of thought into it. My rationale is as follows:
- I use LaTeX to layout the text and code to give a professional look and I am afraid that EBook readers would mess this up.
- The increase in supported formats would create a maintenance headache that would take a large amount of time away from updating the books and working on new books.
- Most critically, reading on an e-reader or iPad is antithetical to the book-open-next-to-code-editor approach the PDF format was chosen to support.
My materials are playbooks intended to be open on the computer, next to a text editor and a command line.
They are not textbooks to be read away from the computer.
Thanks for your interest in my books
I’m sorry that you cannot afford my books or purchase them in your country.
I don’t give away free copies of my books.
I do give away a lot of free material on applied machine learning already.
You can access the best free material here:
Maybe.
I offer a discount on my books to:
- Students
- Teachers
- Retirees
If you fall into one of these groups and would like a discount, please contact me and ask.
Maybe.
I support payment via PayPal and Credit Card.
You may be able to set up a PayPal account that accesses your debit card. I recommend contacting PayPal or reading their documentation.
Sorry no.
I do not support WeChat Pay or Alipay at this stage.
I only support payment via PayPal and Credit Card.
Yes, you can print the purchased PDF books for your own personal interest.
There is no digital rights management (DRM) on the PDFs to prevent you from printing them.
Please do not distribute printed copies of your purchased books.
You can review the table of contents for any book.
I provide two copies of the table of contents for each book on the book’s page.
Specifically:
- A written summary that lists the tutorials/lessons in the book and their order.
- A screenshot of the table of contents taken from the PDF.
If you are having trouble finding the table of contents, search the page for the section titled “Table of Contents”.
Yes.
If you purchase a book or bundle and later decide that you want to upgrade to the super bundle, I can arrange it for you.
Contact me and let me know that you would like to upgrade and what books or bundles you have already purchased and which email address you used to make the purchases.
I will create a special offer code that you can use to get the price of books and bundles purchased so far deducted from the price of the super bundle.
I am happy for you to use parts of my material in the development of your own course material, such as lecture slides for an in person class or homework exercises.
I am not happy if you share my material for free or use it verbatim. This would be copyright infringement.
All code on my site and in my books was developed and provided for educational purposes only. I take no responsibility for the code, what it might do, or how you might use it.
If you use my material to teach, please reference the source, including:
- The Name of the author, e.g. “Jason Brownlee”.
- The Title of the tutorial or book.
- The Name of the website, e.g. “Machine Learning Mastery”.
- The URL of the tutorial or book.
- The Date you accessed or copied the code.
For example:
- Jason Brownlee, Machine Learning Algorithms in Python, Machine Learning Mastery, Available from https://machinelearningmastery.com/machine-learning-with-python/, accessed April 15th, 2018.
Also, if your work is public, contact me, I’d love to see it out of general interest.
Generally no.
I don’t have exercises or assignments in my books.
I do have end-to-end projects in some of the books, but they are in a tutorial format where I lead you through each step.
The book chapters are written as self-contained tutorials with a specific learning outcome. You will learn how to do something at the end of the tutorial.
Some books have a section titled “Extensions” with ideas for how to modify the code in the tutorial in some advanced ways. They are like self-study exercises.
Sorry, new books are not included in your super bundle.
I release new books every few months and develop a new super bundle at those times.
All existing customers will get early access to new books at a discount price.
Note, that you do get free updates to all of the books in your super bundle. This includes bug fixes, changes to APIs and even new chapters sometimes. I send out an email to customers for major book updates or you can contact me any time and ask for the latest version of a book.
No.
I have books that do not require any skill in programming, for example:
Other books do have code examples in a given programming language.
You must know the basics of the programming language, such as how to install the environment and how to write simple programs. I do not teach programming, I teach machine learning for developers.
You do not need to be a good programmer.
That being said, I do offer tutorials on how to setup your environment efficiently and even crash courses on programming languages for developers that may not be familiar with the given language.
No.
My books do not cover the theory or derivations of machine learning methods.
This is by design.
My books are focused on the practical concern of applied machine learning. Specifically, how algorithms work and how to use them effectively with modern open source tools.
If you are interested in the theory and derivations of equations, I recommend a machine learning textbook. Some good examples of machine learning textbooks that cover theory include:
I generally don’t run sales.
If I do have a special, such as around the launch of a new book, I only offer it to past customers and subscribers on my email list.
I do offer book bundles that offer a discount for a collection of related books.
I do offer a discount to students, teachers, and retirees. Contact me to find out about discounts.
Sorry, I don’t have videos.
I only have tutorial lessons and projects in text format.
This is by design. I used to have video content and I found the completion rate much lower.
I want you to put the material into practice. I have found that text-based tutorials are the best way of achieving this. With text-based tutorials you must read, implement and run the code.
With videos, you are passively watching and not required to take any action. Videos are entertainment or infotainment instead of productive learning and work.
After reading and working through the tutorials you are far more likely to use what you have learned.
Yes, I offer a 90-day no questions asked money-back guarantee.
I stand behind my books. They contain my best knowledge on a specific machine learning topic, and each book as been read, tested and used by tens of thousands of readers.
Nevertheless, if you find that one of my Ebooks is a bad fit for you, I will issue a full refund.
There are no physical books, therefore no shipping is required.
All books are EBooks that you can download immediately after you complete your purchase.
I support purchases from any country via PayPal or Credit Card.
Yes.
I recommend using standalone Keras version 2.4 (or higher) running on top of TensorFlow version 2.2 (or higher).
All tutorials on the blog have been updated to use standalone Keras running on top of Tensorflow 2.
All books have been updated to use this same combination.
I do not recommend using Keras as part of TensorFlow 2 yet (e.g. tf.keras). It is too new, new things have issues, and I am waiting for the dust to settle. Standalone Keras has been working for years and continues to work extremely well.
There is one case of tutorials that do not support TensorFlow 2 because the tutorials make use of third-party libraries that have not yet been updated to support TensorFlow 2. Specifically tutorials that use Mask-RCNN for object recognition. Once the third party library has been updated, these tutorials too will be updated.
The book “Long Short-Term Memory Networks with Python” is not focused on time series forecasting, instead, it is focused on the LSTM method for a suite of sequence prediction problems.
The book “Deep Learning for Time Series Forecasting” shows you how to develop MLP, CNN and LSTM models for univariate, multivariate and multi-step time series forecasting problems.
Mini-courses are free courses offered on a range of machine learning topics and made available via email, PDF and blog posts.
Mini-courses are:
- Short, typically 7 days or 14 days in length.
- Terse, typically giving one tip or code snippet per lesson.
- Limited, typically narrow in scope to a few related areas.
Ebooks are provided on many of the same topics providing full training courses on the topics.
Ebooks are:
- Longer, typically 25+ complete tutorial lessons, each taking up to an hour to complete.
- Complete, providing a gentle introduction into each lesson and includes full working code and further reading.
- Broad, covering all of the topics required on the topic to get productive quickly and bring the techniques to your own projects.
The mini-courses are designed for you to get a quick result. If you would like more information or fuller code examples on the topic then you can purchase the related Ebook.
The book “Master Machine Learning Algorithms” is for programmers and non-programmers alike. It teaches you how 10 top machine learning algorithms work, with worked examples in arithmetic, and spreadsheets, not code. The focus is on an understanding on how each model learns and makes predictions.
The book “Machine Learning Algorithms From Scratch” is for programmers that learn by writing code to understand. It provides step-by-step tutorials on how to implement top algorithms as well as how to load data, evaluate models and more. It has less on how the algorithms work, instead focusing exclusively on how to implement each in code.
The two books can support each other.
The books are a concentrated and more convenient version of what I put on the blog.
I design my books to be a combination of lessons and projects to teach you how to use a specific machine learning tool or library and then apply it to real predictive modeling problems.
The books get updated with bug fixes, updates for API changes and the addition of new chapters, and these updates are totally free.
I do put some of the book chapters on the blog as examples, but they are not tied to the surrounding chapters or the narrative that a book offers and do not offer the standalone code files.
With each book, you also get all of the source code files used in the book that you can use as recipes to jump-start your own predictive modeling problems.
My books are playbooks. Not textbooks.
They have no deep explanations of theory, just working examples that are laser-focused on the information that you need to know to bring machine learning to your project.
There is little math, no theory or derivations.
My readers really appreciate the top-down, rather than bottom-up approach used in my material. It is the one aspect I get the most feedback about.
My books are not for everyone, they are carefully designed for practitioners that need to get results, fast.
A code file is provided for each example presented in the book.
Dataset files used in each chapter are also provided with the book.
The code and dataset files are provided as part of your .zip download in a code/ subdirectory. Code and datasets are organized into subdirectories, one for each chapter that has a code example.
If you have misplaced your .zip download, you can contact me and I can send an updated purchase receipt email with a link to download your package.
Ebooks can be purchased from my website directly.
- First, find the book or bundle that you wish to purchase, you can see the full catalog here:
- Click on the book or bundle that you would like to purchase to go to the book’s details page.
- Click the “Buy Now” button for the book or bundle to go to the shopping cart page.
- Fill in the shopping cart with your details and payment details, and click the “Place Order” button.
- After completing the purchase you will be emailed a link to download your book or bundle.
All prices are in US dollars (USD).
Books can be purchased with PayPal or Credit Card.
All prices on Machine Learning Mastery are in US dollars.
Payments can be made by using either PayPal or a Credit Card that supports international payments (e.g. most credit cards).
You do not have to explicitly convert money from your currency to US dollars.
Currency conversion is performed automatically when you make a payment using PayPal or Credit Card.
After filling out and submitting your order form, you will be able to download your purchase immediately.
Your web browser will be redirected to a webpage where you can download your purchase.
You will also receive an email with a link to download your purchase.
If you lose the email or the link in the email expires, contact me and I will resend the purchase receipt email with an updated download link.
After you complete your purchase you will receive an email with a link to download your bundle.
The download will include the book or books and any bonus material.
To use a discount code, also called an offer code, or discount coupon when making a purchase, follow these steps:
1. Enter the discount code text into the field named “Discount Coupon” on the checkout page.
Note, if you don’t see a field called “Discount Coupon” on the checkout page, it means that that product does not support discounts.
2. Click the “Apply” button.
3. You will then see a message that the discount was applied successfully to your order.
Note, if the discount code that you used is no longer valid, you will see a message that the discount was not successfully applied to your order.
There are no physical books, therefore no shipping is required.
All books are EBooks that you can download immediately after you complete your purchase.
I recommend reading one chapter per day.
Momentum is important.
Some readers finish a book in a weekend.
Most readers finish a book in a few weeks by working through it during nights and weekends.
You will get your book immediately.
After you complete and submit the payment form, you will be immediately redirected to a webpage with a link to download your purchase.
You will also immediately be sent an email with a link to download your purchase.
What order should you read the books?
That is a great question, my best suggestions are as follows:
- Consider starting with a book on a topic that you are most excited about.
- Consider starting with a book on a topic that you can apply on a project immediately.
Also, consider that you don’t need to read all of the books, perhaps a subset of the books will get you the skills you need or want.
Nevertheless, one suggested order for reading the books is as follows:
-
- Probability for Machine Learning
- Statistical Methods for Machine Learning
- Linear Algebra for Machine Learning
- Optimization for Machine Learning
- Calculus for Machine Learning
- The Beginner’s Guide to Data Science
- Master Machine Learning Algorithms
- Machine Learning Algorithms From Scratch
- Python for Machine Learning
- Machine Learning Mastery With Weka
- Machine Learning Mastery With Python
- Machine Learning Mastery With R
- Data Preparation for Machine Learning
- Imbalanced Classification With Python
- Time Series Forecasting With Python
- Ensemble Learning Algorithms With Python
- XGBoost With Python
- Deep Learning With Python
- Deep Learning with PyTorch
- Long Short-Term Memory Networks with Python
- Deep Learning for Natural Language Processing
- Deep Learning for Computer Vision
- Machine Learning in Open CV
- Deep Learning for Time Series Forecasting
- Better Deep Learning
- Generative Adversarial Networks with Python
- Building Transformer Models with Attention
- Productivity with ChatGPT (this book can be read in any order)
I hope that helps.
Sorry, I do not have a license to purchase my books or bundles for libraries.
The books are for individual use only.
Generally, no.
Multi-seat licenses create a bit of a maintenance nightmare for me, sorry. It takes time away from reading, writing and helping my readers.
If you have a big order, such as for a class of students or a large team, please contact me and we will work something out.
I update the books frequently and you can access the latest version of a book at any time.
In order to get the latest version of a book, contact me directly with your order number or purchase email address and I can resend your purchase receipt email with an updated download link.
I do not maintain a public change log or errata for the changes in the book, sorry.
There are no physical books, therefore no delivery is required.
All books are Ebooks in PDF format that you can download immediately after you complete your purchase.
You will receive an email with a link to download your purchase. You can also contact me any time to get a new download link.
I support purchases from any country via PayPal or Credit Card.
My best advice is to start with a book on a topic that you can use immediately.
Baring that, pick a topic that interests you the most.
If you are unsure, perhaps try working through some of the free tutorials to see what area that you gravitate towards.
Generally, I recommend focusing on the process of working through a predictive modeling problem end-to-end:
I have three books that show you how to do this, with three top open source platforms:
- Master Machine Learning With Weka (no programming)
- Master Machine Learning With R (caret)
- Master Machine Learning With Python (pandas and scikit-learn)
These are great places to start.
You can always circle back and pick-up a book on algorithms later to learn more about how specific methods work in greater detail.
Thanks for your interest.
You can see the full catalog of my books and bundles here:
Thanks for asking.
I try not to plan my books too far into the future. I try to write about the topics that I am asked about the most or topics where I see the most misunderstanding.
If you would like me to write more about a topic, I would love to know.
Contact me directly and let me know the topic and even the types of tutorials you would love for me to write.
Contact me and let me know the email address (or email addresses) that you think you used to make purchases.
I can look up what purchases you have made and resend purchase receipts to you so that you can redownload your books and bundles.
All prices are in US Dollars (USD).
All currency conversion is handled by PayPal for PayPal purchases, or by Stripe and your bank for credit card purchases.
It is possible that your link to download your purchase will expire after a few days.
This is a security precaution.
Please contact me and I will resend you purchase receipt with an updated download link.
The book “Deep Learning With Python” could be a prerequisite to”Long Short-Term Memory Networks with Python“. It teaches you how to get started with Keras and how to develop your first MLP, CNN and LSTM.
The book “Long Short-Term Memory Networks with Python” goes deep on LSTMs and teaches you how to prepare data, how to develop a suite of different LSTM architectures, parameter tuning, updating models and more.
Both books focus on deep learning in Python using the Keras library.
The book “Long Short-Term Memory Networks in Python” focuses on how to develop a suite of different LSTM networks for sequence prediction, in general.
The book “Deep Learning for Time Series Forecasting” focuses on how to use a suite of different deep learning models (MLPs, CNNs, LSTMs, and hybrids) to address a suite of different time series forecasting problems (univariate, multivariate, multistep and combinations).
The LSTM book teaches LSTMs only and does not focus on time series. The Deep Learning for Time Series book focuses on time series and teaches how to use many different models including LSTMs.
The book “Long Short-Term Memory Networks With Python” focuses on how to implement different types of LSTM models.
The book “Deep Learning for Natural Language Processing” focuses on how to use a variety of different networks (including LSTMs) for text prediction problems.
The LSTM book can support the NLP book, but it is not a prerequisite.
You may need a business or corporate tax number for “Machine Learning Mastery“, the company, for your own tax purposes. This is common in EU companies for example.
The Machine Learning Mastery company is operated out of Puerto Rico.
As such, the company does not have a VAT identification number for the EU or similar for your country or regional area.
The company does have a Company Number. The details are as follows:
- Company Name: Zeus LLC
- Company Number: 421867-1511
There are no code examples in “Master Machine Learning Algorithms“, therefore no programming language is used.
Algorithms are described and their working is summarized using basic arithmetic. The algorithm behavior is also demonstrated in excel spreadsheets, that are available with the book.
It is a great book for learning how algorithms work, without getting side-tracked with theory or programming syntax.
If you are interested in learning about machine learning algorithms by coding them from scratch (using the Python programming language), I would recommend a different book:
I write the content for the books (words and code) using a text editor, specifically sublime.
I typeset the books and create a PDF using LaTeX.
All of the books have been tested and work with Python 3 (e.g. 3.5 or 3.6).
Most of the books have also been tested and work with Python 2.7.
Where possible, I recommend using the latest version of Python 3.
After you fill in the order form and submit it, two things will happen:
- You will be redirected to a webpage where you can download your purchase.
- You will be sent an email (to the email address used in the order form) with a link to download your purchase.
The redirect in the browser and the email will happen immediately after you complete the purchase.
You can download your purchase from either the webpage or the email.
If you cannot find the email, perhaps check other email folders, such as the “spam” folder?
If you have any concerns, contact me and I can resend your purchase receipt email with the download link.
I do test my tutorials and projects on the blog first. It’s like the early access to ideas, and many of them do not make it to my training.
Much of the material in the books appeared in some form on my blog first and is later refined, improved and repackaged into a chapter format. I find this helps greatly with quality and bug fixing.
The books provide a more convenient packaging of the material, including source code, datasets and PDF format. They also include updates for new APIs, new chapters, bug and typo fixing, and direct access to me for all the support and help I can provide.
I believe my books offer thousands of dollars of education for tens of dollars each.
They are months if not years of experience distilled into a few hundred pages of carefully crafted and well-tested tutorials.
I think they are a bargain for professional developers looking to rapidly build skills in applied machine learning or use machine learning on a project.
Also, what are skills in machine learning worth to you? to your next project? and you’re current or next employer?
Nevertheless, the price of my books may appear expensive if you are a student or if you are not used to the high salaries for developers in North America, Australia, UK and similar parts of the world. For that, I am sorry.
Discounts
I do offer discounts to students, teachers and retirees.
Please contact me to find out more.
Free Material
I offer a ton of free content on my blog, you can get started with my best free material here:
About my Books
My books are playbooks.
They are intended for developers who want to know how to use a specific library to actually solve problems and deliver value at work.
- My books guide you only through the elements you need to know in order to get results.
- My books are in PDF format and come with code and datasets, specifically designed for you to read and work-through on your computer.
- My books give you direct access to me via email (what other books offer that?)
- My books are a tiny business expense for a professional developer that can be charged to the company and is tax deductible in most regions.
Very few training materials on machine learning are focused on how to get results.
The vast majority are about repeating the same math and theory and ignore the one thing you really care about: how to use the methods on a project.
Comparison to Other Options
Let me provide some context for you on the pricing of the books:
There are free videos on youtube and tutorials on blogs.
- Great, I encourage you to use them, including my own free tutorials.
There are very cheap video courses that teach you one or two tricks with an API.
- My books teach you how to use a library to work through a project end-to-end and deliver value, not just a few tricks
A textbook on machine learning can cost $50 to $100.
- All of my books are cheaper than the average machine learning textbook, and I expect you may be more productive, sooner.
A bootcamp or other in-person training can cost $1000+ dollars and last for days to weeks.
- A bundle of all of my books is far cheaper than this, they allow you to work at your own pace, and the bundle covers more content than the average bootcamp.
Sorry, my books are not available on websites like Amazon.com.
I carefully decided to not put my books on Amazon for a number of reasons:
- Amazon takes 65% of the sale price of self-published books, which would put me out of business.
- Amazon offers very little control over the sales page and shopping cart experience.
- Amazon does not allow me to contact my customers via email and offer direct support and updates.
- Amazon does not allow me to deliver my book to customers as a PDF, the preferred format for my customers to read on the screen.
I hope that helps you understand my rationale.
I am sorry to hear that you’re having difficulty purchasing a book or bundle.
I use Stripe for Credit Card and PayPal services to support secure and encrypted payment processing on my website.
Some common problems when customers have a problem include:
- Perhaps you can double check that your details are correct, just in case of a typo?
- Perhaps you could try a different payment method, such as PayPal or Credit Card?
- Perhaps you’re able to talk to your bank, just in case they blocked the transaction?
I often see customers trying to purchase with a domestic credit card or debit card that does not allow international purchases. This is easy to overcome by talking to your bank.
If you’re still having difficulty, please contact me and I can help investigate further.
When you purchase a book from my website and later review your bank statement, it is possible that you may see an additional small charge of one or two dollars.
The charge does not come from my website or payment processor.
Instead, the charge was added by your bank, credit card company, or financial institution. It may be because your bank adds an additional charge for online or international transactions.
This is rare but I have seen this happen once or twice before, often with credit cards used by enterprise or large corporate institutions.
My advice is to contact your bank or financial institution directly and ask them to explain the cause of the additional charge.
If you would like a copy of the payment transaction from my side (e.g. a screenshot from the payment processor), or a PDF tax invoice, please contact me directly.
I give away a lot of content for free. Most of it in fact.
It is important to me to help students and practitioners that are not well off, hence the enormous amount of free content that I provide.
You can access the free content:
I have thought very hard about this and I sell machine learning Ebooks for a few important reasons:
- I use the revenue to support the site and all the non-paying customers.
- I use the revenue to support my family so that I can continue to create content.
- Practitioners that pay for tutorials are far more likely to work through them and learn something.
- I target my books towards working professionals that are more likely to afford the materials.
Yes.
All updates to the book or books in your purchase are free.
Books are usually updated once every few months to fix bugs, typos and keep abreast of API changes.
Contact me anytime and check if there have been updates. Let me know what version of the book you have (version is listed on the copyright page).
Yes.
Please contact me anytime with questions about machine learning or the books.
One question at a time please.
Also, each book has a final chapter on getting more help and further reading and points to resources that you can use to get more help.
Yes, the books can help you get a job, but indirectly.
Getting a job is up to you.
It is a matching problem between an organization looking for someone to fill a role and you with your skills and background.
That being said, there are companies that are more interested in the value that you can provide to the business than the degrees that you have. Often, these are smaller companies and start-ups.
You can focus on providing value with machine learning by learning and getting very good at working through predictive modeling problems end-to-end. You can show this skill by developing a machine learning portfolio of completed projects.
My books are specifically designed to help you toward these ends. They teach you exactly how to use open source tools and libraries to get results in a predictive modeling project.
Do you have another question?