Archive | Better Deep Learning

Example of Local and Global Minima on an Error Surface

Why Training a Neural Network Is Hard

Or, Why Stochastic Gradient Descent Is Used to Train Neural Networks. Fitting a neural network involves using a training dataset to update the model weights to create a good mapping of inputs to outputs. This training process is solved using an optimization algorithm that searches through a space of possible values for the neural network […]

Continue Reading 12
Example of Train and Validation Learning Curves Showing a Training Dataset That May Be too Small Relative to the Validation Dataset

How to use Learning Curves to Diagnose Machine Learning Model Performance

A learning curve is a plot of model learning performance over experience or time. Learning curves are a widely used diagnostic tool in machine learning for algorithms that learn from a training dataset incrementally. The model can be evaluated on the training dataset and on a hold out validation dataset after each update during training […]

Continue Reading 65
Practical Recommendations for Deep Learning Neural Network Practitioners

Recommendations for Deep Learning Neural Network Practitioners

Deep learning neural networks are relatively straightforward to define and train given the wide adoption of open source libraries. Nevertheless, neural networks remain challenging to configure and train. In his 2012 paper titled “Practical Recommendations for Gradient-Based Training of Deep Architectures” published as a preprint and a chapter of the popular 2012 book “Neural Networks: […]

Continue Reading 6
8 Tricks for Configuring Backpropagation to Train Better Neural Networks, Faster

8 Tricks for Configuring Backpropagation to Train Better Neural Networks

Neural network models are trained using stochastic gradient descent and model weights are updated using the backpropagation algorithm. The optimization solved by training a neural network model is very challenging and although these algorithms are widely used because they perform so well in practice, there are no guarantees that they will converge to a good […]

Continue Reading 4
Framework for Better Deep Learning

Framework for Better Deep Learning

Modern deep learning libraries such as Keras allow you to define and start fitting a wide range of neural network models in minutes with just a few lines of code. Nevertheless, it is still challenging to configure a neural network to get good performance on a new predictive modeling problem. The challenge of getting good […]

Continue Reading 2