Logistic regression is a model for binary classification predictive modeling. The parameters of a logistic regression model can be estimated […]

# Archive | Probability

## A Gentle Introduction to Linear Regression With Maximum Likelihood Estimation

Linear regression is a classical model for predicting a numerical quantity. The parameters of a linear regression model can be […]

## A Gentle Introduction to Maximum Likelihood Estimation for Machine Learning

Density estimation is the problem of estimating the probability distribution for a sample of observations from a problem domain. There […]

## A Gentle Introduction to Cross-Entropy for Machine Learning

Cross-entropy is commonly used in machine learning as a loss function. Cross-entropy is a measure from the field of information […]

## How to Calculate the KL Divergence for Machine Learning

It is often desirable to quantify the difference between probability distributions for a given random variable. This occurs frequently in […]

## Information Gain and Mutual Information for Machine Learning

Information gain calculates the reduction in entropy or surprise from transforming a dataset in some way. It is commonly used […]

## A Gentle Introduction to Information Entropy

Information theory is a subfield of mathematics concerned with transmitting data across a noisy channel. A cornerstone of information theory […]

## A Gentle Introduction to Bayesian Belief Networks

Probabilistic models can define relationships between variables and be used to calculate probabilities. For example, fully conditional models may require […]

## How to Implement Bayesian Optimization from Scratch in Python

In this tutorial, you will discover how to implement the Bayesian Optimization algorithm for complex optimization problems. Global optimization is […]

## How to Develop a Naive Bayes Classifier from Scratch in Python

Classification is a predictive modeling problem that involves assigning a label to a given input data sample. The problem of […]