Surface Plot of a Two-Dimensional Objective Function

Visualization for Function Optimization in Python

Function optimization involves finding the input that results in the optimal value from an objective function. Optimization algorithms navigate the search space of input variables in order to locate the optima, and both the shape of the objective function and behavior of the algorithm in the search space are opaque on real-world problems. As such, […]

Continue Reading 2
Algorithms for Optimization

3 Books on Optimization for Machine Learning

Optimization is a field of mathematics concerned with finding a good or best solution among many candidates. It is an important foundational topic required in machine learning as most machine learning algorithms are fit on historical data using an optimization algorithm. Additionally, broader problems, such as model selection and hyperparameter tuning, can also be framed […]

Continue Reading 6
Line Plot of a Non-Convex Objective Function with Optima Marked

Univariate Function Optimization in Python

How to Optimize a Function with One Variable? Univariate function optimization involves finding the input to a function that results in the optimal output from an objective function. This is a common procedure in machine learning when fitting a model with one parameter or tuning a model that has a single hyperparameter. An efficient algorithm […]

Continue Reading 6
Semi-Supervised Learning With Label Spreading

Semi-Supervised Learning With Label Spreading

Semi-supervised learning refers to algorithms that attempt to make use of both labeled and unlabeled training data. Semi-supervised learning algorithms are unlike supervised learning algorithms that are only able to learn from labeled training data. A popular approach to semi-supervised learning is to create a graph that connects examples in the training dataset and propagates […]

Continue Reading 16
Box and Whisker Plots of L2 Penalty Configuration vs. Accuracy for Multinomial Logistic Regression

Multinomial Logistic Regression With Python

Multinomial logistic regression is an extension of logistic regression that adds native support for multi-class classification problems. Logistic regression, by default, is limited to two-class classification problems. Some extensions like one-vs-rest can allow logistic regression to be used for multi-class classification problems, although they require that the classification problem first be transformed into multiple binary […]

Continue Reading 10
Semi-Supervised Learning With Label Propagation

Semi-Supervised Learning With Label Propagation

Semi-supervised learning refers to algorithms that attempt to make use of both labeled and unlabeled training data. Semi-supervised learning algorithms are unlike supervised learning algorithms that are only able to learn from labeled training data. A popular approach to semi-supervised learning is to create a graph that connects examples in the training dataset and propagate […]

Continue Reading 12