Developing a neural network predictive model for a new dataset can be challenging. One approach is to first inspect the dataset and develop ideas for what models might work, then explore the learning dynamics of simple models on the dataset, then finally develop and tune a model for the dataset with a robust test harness. […]
The Nelder-Mead optimization algorithm is a widely used approach for non-differentiable objective functions. As such, it is generally referred to as a pattern search algorithm and is used as a local or global search procedure, challenging nonlinear and potentially noisy and multimodal function optimization problems. In this tutorial, you will discover the Nelder-Mead optimization algorithm. […]
Recommender systems may be the most common type of predictive model that the average person may encounter. They provide the basis for recommendations on services such as Amazon, Spotify, and Youtube. Recommender systems are a huge daunting topic if you’re just getting started. There is a myriad of data preparation techniques, algorithms, and model evaluation […]
Regression refers to predictive modeling problems that involve predicting a numeric value. It is different from classification that involves predicting a class label. Unlike classification, you cannot use classification accuracy to evaluate the predictions made by a regression model. Instead, you must use error metrics specifically designed for evaluating predictions made on regression problems. In […]
Activation functions are a critical part of the design of a neural network. The choice of activation function in the hidden layer will control how well the network model learns the training dataset. The choice of activation function in the output layer will define the type of predictions the model can make. As such, a […]
Function optimization involves finding the input that results in the optimal value from an objective function. Optimization algorithms navigate the search space of input variables in order to locate the optima, and both the shape of the objective function and behavior of the algorithm in the search space are opaque on real-world problems. As such, […]
Gradient descent is an optimization algorithm that follows the negative gradient of an objective function in order to locate the minimum of the function. A limitation of gradient descent is that a single step size (learning rate) is used for all input variables. Extensions to gradient descent like AdaGrad and RMSProp update the algorithm to […]
Optimization is a field of mathematics concerned with finding a good or best solution among many candidates. It is an important foundational topic required in machine learning as most machine learning algorithms are fit on historical data using an optimization algorithm. Additionally, broader problems, such as model selection and hyperparameter tuning, can also be framed […]
How to Optimize a Function with One Variable? Univariate function optimization involves finding the input to a function that results in the optimal output from an objective function. This is a common procedure in machine learning when fitting a model with one parameter or tuning a model that has a single hyperparameter. An efficient algorithm […]
Applied machine learning is typically focused on finding a single model that performs well or best on a given dataset. Effective use of the model will require appropriate preparation of the input data and hyperparameter tuning of the model. Collectively, the linear sequence of steps required to prepare the data, tune the model, and transform […]
