Make the Confusion Matrix Less Confusing. A confusion matrix is a technique for summarizing the performance of a classification algorithm. Classification accuracy alone can be misleading if you have an unequal number of observations in each class or if you have more than two classes in your dataset. Calculating a confusion matrix can give you […]

# Archive | Code Machine Learning Algorithms From Scratch

## How to Implement Stacked Generalization (Stacking) From Scratch With Python

Code a Stacking Ensemble From Scratch in Python, Step-by-Step. Ensemble methods are an excellent way to improve predictive performance on your machine learning problems. Stacked Generalization or stacking is an ensemble technique that uses a new model to learn how to best combine the predictions from two or more models trained on your dataset. In […]

## How to Implement Random Forest From Scratch in Python

Decision trees can suffer from high variance which makes their results fragile to the specific training data used. Building multiple models from samples of your training data, called bagging, can reduce this variance, but the trees are highly correlated. Random Forest is an extension of bagging that in addition to building trees based on multiple […]

## How to Implement Bagging From Scratch With Python

Decision trees are a simple and powerful predictive modeling technique, but they suffer from high-variance. This means that trees can get very different results given different training data. A technique to make decision trees more robust and to achieve better performance is called bootstrap aggregation or bagging for short. In this tutorial, you will discover […]

## How To Implement The Decision Tree Algorithm From Scratch In Python

Decision trees are a powerful prediction method and extremely popular. They are popular because the final model is so easy to understand by practitioners and domain experts alike. The final decision tree can explain exactly why a specific prediction was made, making it very attractive for operational use. Decision trees also provide the foundation for […]

## How to Implement the Backpropagation Algorithm From Scratch In Python

The backpropagation algorithm is the classical feed-forward artificial neural network. It is the technique still used to train large deep learning networks. In this tutorial, you will discover how to implement the backpropagation algorithm from scratch with Python. After completing this tutorial, you will know: How to forward-propagate an input to calculate an output. How […]

## How To Implement Learning Vector Quantization From Scratch With Python

A limitation of k-Nearest Neighbors is that you must keep a large database of training examples in order to make predictions. The Learning Vector Quantization algorithm addresses this by learning a much smaller subset of patterns that best represent the training data. In this tutorial, you will discover how to implement the Learning Vector Quantization […]

## How To Implement The Perceptron Algorithm From Scratch In Python

The Perceptron algorithm is the simplest type of artificial neural network. It is a model of a single neuron that can be used for two-class classification problems and provides the foundation for later developing much larger networks. In this tutorial, you will discover how to implement the Perceptron algorithm from scratch with Python. After completing […]

## How To Implement Logistic Regression With Stochastic Gradient Descent From Scratch With Python

Logistic regression is the go-to linear classification algorithm for two-class problems. It is easy to implement, easy to understand and gets great results on a wide variety of problems, even when the expectations the method has of your data are violated. In this tutorial, you will discover how to implement logistic regression with stochastic gradient […]

## How to Implement Linear Regression With Stochastic Gradient Descent From Scratch With Python

The core of many machine learning algorithms is optimization. Optimization algorithms are used by machine learning algorithms to find a good set of model parameters given a training dataset. The most common optimization algorithm used in machine learning is stochastic gradient descent. In this tutorial, you will discover how to implement stochastic gradient descent to […]