Author Archive | Stefania Cristina

neural_networks_cover

Calculus in Action: Neural Networks

An artificial neural network is a computational model that approximates a mapping between inputs and outputs.  It is inspired by the structure of the human brain, in that it is similarly composed of a network of interconnected neurons that propagate information upon receiving sets of stimuli from neighbouring neurons. Training a neural network involves a […]

Continue Reading 13
more_chain_rule_cover

The Chain Rule of Calculus – Even More Functions

The chain rule is an important derivative rule that allows us to work with composite functions. It is essential in understanding the workings of the backpropagation algorithm, which applies the chain rule extensively in order to calculate the error gradient of the loss function with respect to each weight of a neural network. We will […]

Continue Reading 0
chain_rule_cover

The Chain Rule of Calculus for Univariate and Multivariate Functions

The chain rule allows us to find the derivative of composite functions. It is computed extensively by the backpropagation algorithm, in order to train feedforward neural networks. By applying the chain rule in an efficient manner while following a specific order of operations, the backpropagation algorithm calculates the error gradient of the loss function with […]

Continue Reading 6
laplacian_cover

A Gentle Introduction to the Laplacian

The Laplace operator was first applied to the study of celestial mechanics, or the motion of objects in outer space, by Pierre-Simon de Laplace, and as such has been named after him.  The Laplace operator has since been used to describe many different phenomena, from electric potentials, to the diffusion equation for heat and fluid […]

Continue Reading 5
jacobian_cover

A Gentle Introduction to the Jacobian

In the literature, the term Jacobian is often interchangeably used to refer to both the Jacobian matrix or its determinant.  Both the matrix and the determinant have useful and important applications: in machine learning, the Jacobian matrix aggregates the partial derivatives that are necessary for backpropagation; the determinant is useful in the process of changing […]

Continue Reading 21
higher_order_cover

Higher-Order Derivatives

Higher-order derivatives can capture information about a function that first-order derivatives on their own cannot capture.  First-order derivatives can capture important information, such as the rate of change, but on their own they cannot distinguish between local minima or maxima, where the rate of change is zero for both. Several optimization algorithms address this limitation […]

Continue Reading 2
integral_cover

Differential and Integral Calculus – Differentiate with Respect to Anything

Integral calculus was one of the greatest discoveries of Newton and Leibniz. Their work independently led to the proof, and recognition of the importance of the fundamental theorem of calculus, which linked integrals to derivatives. With the discovery of integrals, areas and volumes could thereafter be studied.  Integral calculus is the second half of the […]

Continue Reading 4
applications_cover

Applications of Derivatives

The derivative defines the rate at which one variable changes with respect to another.  It is an important concept that comes in extremely useful in many applications: in everyday life, the derivative can tell you at which speed you are driving, or help you predict fluctuations on the stock market; in machine learning, derivatives are […]

Continue Reading 2