Archive | Ensemble Learning

Essence of Bootstrap Aggregation Ensembles

Essence of Bootstrap Aggregation Ensembles

Bootstrap aggregation, or bagging, is a popular ensemble method that fits a decision tree on different bootstrap samples of the training dataset. It is simple to implement and effective on a wide range of problems, and importantly, modest extensions to the technique result in ensemble methods that are among some of the most powerful techniques, […]

Continue Reading
A Gentle Introduction to Ensemble Diversity for Machine Learning

A Gentle Introduction to Ensemble Diversity for Machine Learning

Ensemble learning combines the predictions from machine learning models for classification and regression. We pursue using ensemble methods to achieve improved predictive performance, and it is this improvement over any of the contributing models that defines whether an ensemble is good or not. A property that is present in a good ensemble is the diversity […]

Continue Reading
A Gentle Introduction to Multiple-Model Machine Learning

A Gentle Introduction to Multiple-Model Machine Learning

An ensemble learning method involves combining the predictions from multiple contributing models. Nevertheless, not all techniques that make use of multiple machine learning models are ensemble learning algorithms. It is common to divide a prediction problem into subproblems. For example, some problems naturally subdivide into independent but related subproblems and a machine learning model can […]

Continue Reading
Essence of Boosting Ensembles for Machine Learning

Essence of Boosting Ensembles for Machine Learning

Boosting is a powerful and popular class of ensemble learning techniques. Historically, boosting algorithms were challenging to implement, and it was not until AdaBoost demonstrated how to implement boosting that the technique could be used effectively. AdaBoost and modern gradient boosting work by sequentially adding models that correct the residual prediction errors of the model. […]

Continue Reading
Box and Whisker Plots of Classification Accuracy for Standalone Machine Learning Models

Growing and Pruning Ensembles in Python

Ensemble member selection refers to algorithms that optimize the composition of an ensemble. This may involve growing an ensemble from available models or pruning members from a fully defined ensemble. The goal is often to reduce the model or computational complexity of an ensemble with little or no effect on the performance of an ensemble, […]

Continue Reading
Box and Whisker Plots of Accuracy Distributions for k Values in KNORA-U

Dynamic Ensemble Selection (DES) for Classification in Python

Dynamic ensemble selection is an ensemble learning technique that automatically selects a subset of ensemble members just-in-time when making a prediction. The technique involves fitting multiple machine learning models on the training dataset, then selecting the models that are expected to perform best when making a prediction for a specific new example, based on the […]

Continue Reading