Deep learning neural networks are challenging to configure and train. There are decades of tips and tricks spread across hundreds of research papers, source code, and in the heads of academics and practitioners. The book “Neural Networks: Tricks of the Trade” originally published in 1998 and updated in 2012 at the cusp of the deep […]
Weight regularization provides an approach to reduce the overfitting of a deep learning neural network model on the training data and improve the performance of the model on new data, such as the holdout test set. There are multiple types of weight regularization, such as L1 and L2 vector norms, and each requires a hyperparameter […]
Recurrent neural networks can also be used as generative models. This means that in addition to being used for predictive models (making predictions), they can learn the sequences of a problem and then generate entirely new plausible sequences for the problem domain. Generative models like this are useful not only to study how well a […]
