When it comes to sequential or time series data, traditional feedforward networks cannot be used for learning and prediction. A […]
Training-validation-test split and cross-validation done right
One crucial step in machine learning is the choice of model. A suitable model with suitable hyperparameter is the key […]
The Attention Mechanism from Scratch
The attention mechanism was introduced to improve the performance of the encoder-decoder model for machine translation. The idea behind the […]
A Gentle Introduction to Particle Swarm Optimization
Particle swarm optimization (PSO) is one of the bio-inspired algorithms and it is a simple one to search for an […]
What is Attention?
Attention is becoming increasingly popular in machine learning, but what makes it such an attractive concept? What is the relationship […]
A Bird’s Eye View of Research on Attention
Attention is a concept that is scientifically studied across multiple disciplines, including psychology, neuroscience and, more recently, machine learning. While […]
Lagrange Multiplier Approach with Inequality Constraints
In a previous post, we introduced the method of Lagrange multipliers to find local minima or local maxima of a […]
A Gentle Introduction To Sigmoid Function
Whether you implement a neural network yourself or you use a built in library for neural network learning, it is […]
Calculus in Action: Neural Networks
An artificial neural network is a computational model that approximates a mapping between inputs and outputs. It is inspired by […]
A Gentle Introduction to Taylor Series
A Gentle Introduction to Taylor Series Taylor series expansion is an awesome concept, not only the world of mathematics, but […]