[New Book] Click to get Deep Learning with PyTorch!
Use the offer code 20offearlybird to get 20% off.

Archive | Optimization

How to Choose an Optimization Algorithm

How to Choose an Optimization Algorithm

Optimization is the problem of finding a set of inputs to an objective function that results in a maximum or minimum function evaluation. It is the challenging problem that underlies many machine learning algorithms, from fitting logistic regression models to training artificial neural networks. There are perhaps hundreds of popular optimization algorithms, and perhaps tens […]

Continue Reading 37
How to Manually Optimize Neural Network Models

How to Manually Optimize Neural Network Models

Deep learning neural network models are fit on training data using the stochastic gradient descent optimization algorithm. Updates to the weights of the model are made, using the backpropagation of error algorithm. The combination of the optimization and weight update algorithm was carefully chosen and is the most efficient approach known to fit neural networks. […]

Continue Reading 13
Foundations of Genetic Programming

Books on Genetic Programming

Genetic Programming (GP) is an algorithm for evolving programs to solve specific well-defined problems. It is a type of automatic programming intended for challenging problems where the task is well defined and solutions can be checked easily at a low cost, although the search space of possible solutions is vast, and there is little intuition […]

Continue Reading 12
Response Surface of Objective Function With Sequence of Best Solutions Plotted as Black Dots

Stochastic Hill Climbing in Python from Scratch

Stochastic Hill climbing is an optimization algorithm. It makes use of randomness as part of the search process. This makes the algorithm appropriate for nonlinear objective functions where other local search algorithms do not operate well. It is also a local search algorithm, meaning that it modifies a single solution and searches the relatively local […]

Continue Reading 7
Curve Fitting With Python

Curve Fitting With Python

Curve fitting is a type of optimization that finds an optimal set of parameters for a defined function that best fits a given set of observations. Unlike supervised learning, curve fitting requires that you define the function that maps examples of inputs to outputs. The mapping function, also called the basis function can have any […]

Continue Reading 72