Calculating the average of a variable or a list of numbers is a common operation in machine learning.
It is an operation you may use every day either directly, such as when summarizing data, or indirectly, such as a smaller step in a larger procedure when fitting a model.
The average is a synonym for the mean, a number that represents the most likely value from a probability distribution. As such, there are multiple different ways to calculate the mean based on the type of data that you’re working with.
This can trip you up if you use the wrong mean for your data. You may also enter some of these more exotic calculations of mean values when using performance metrics to evaluate your model, such as the G-mean or the F-Measure.
In this tutorial, you will discover the difference between the arithmetic mean, the geometric mean, and the harmonic mean.
After completing this tutorial, you will know:
The central tendency summarizes the most likely value for a variable, and the average is the common name for the calculation of the mean.
The arithmetic mean is appropriate if the values have the same units, whereas the geometric mean is appropriate if the values have differing units.
The harmonic mean is appropriate if the data values are ratios of two variables with different measures, called rates.
Kick-start your project with my new book Statistics for Machine Learning, including step-by-step tutorials and the Python source code files for all examples.
Let’s get started.
Arithmetic, Geometric, and Harmonic Means for Machine Learning Photo by Ray in Manila, some rights reserved.
Tutorial Overview
This tutorial is divided into five parts; they are:
What Is the Average?
Arithmetic Mean
Geometric Mean
Harmonic Mean
How to Choose the Correct Mean?
What Is the Average?
The central tendency is a single number that represents the most common value for a list of numbers.
More technically, it is the value that has the highest probability from the probability distribution that describes all possible values that a variable may have.
There are many ways to calculate the central tendency for a data sample, such as the mean which is calculated from the values, the mode, which is the most common value in the data distribution, or the median, which is the middle value if all values in the data sample were ordered.
The average is the common term for the mean. They can be used interchangeably.
The mean is different from the median and the mode in that it is a measure of the central tendency that is calculated from the data. As such, there are different ways to calculate the mean based on the type of data.
Three common types of mean calculations that you may encounter are the arithmetic mean, the geometric mean, and the harmonic mean. There are other means, and many more central tendency measures, but these three means are perhaps the most common (e.g. the so-called Pythagorean means).
Let’s take a closer look at each calculation of the mean in turn.
Arithmetic Mean
The arithmetic mean is calculated as the sum of the values divided by the total number of values, referred to as N.
Arithmetic Mean = (x1 + x2 + … + xN) / N
A more convenient way to calculate the arithmetic mean is to calculate the sum of the values and to multiply it by the reciprocal of the number of values (1 over N); for example:
Arithmetic Mean = (1/N) * (x1 + x2 + … + xN)
The arithmetic mean is appropriate when all values in the data sample have the same units of measure, e.g. all numbers are heights, or dollars, or miles, etc.
When calculating the arithmetic mean, the values can be positive, negative, or zero.
The arithmetic mean can be easily distorted if the sample of observations contains outliers (a few values far away in feature space from all other values), or for data that has a non-Gaussian distribution (e.g. multiple peaks, a so-called multi-modal probability distribution).
The arithmetic mean is useful in machine learning when summarizing a variable, e.g. reporting the most likely value. This is more meaningful when a variable has a Gaussian or Gaussian-like data distribution.
The example below demonstrates how to calculate the arithmetic mean for a list of 10 numbers.
1
2
3
4
5
6
7
# example of calculating the arithmetic mean
from numpy import mean
# define the dataset
data=[0,1,2,3,4,5,6,7,8,9]
# calculate the mean
result=mean(data)
print('Arithmetic Mean: %.3f'%result)
Running the example calculates the arithmetic mean and reports the result.
1
Arithmetic Mean: 4.500
Geometric Mean
The geometric mean is calculated as the N-th root of the product of all values, where N is the number of values.
Geometric Mean = N-root(x1 * x2 * … * xN)
For example, if the data contains only two values, the square root of the product of the two values is the geometric mean. For three values, the cube-root is used, and so on.
The geometric mean is appropriate when the data contains values with different units of measure, e.g. some measure are height, some are dollars, some are miles, etc.
The geometric mean does not accept negative or zero values, e.g. all values must be positive.
One common example of the geometric mean in machine learning is in the calculation of the so-called G-Mean (geometric mean) metric that is a model evaluation metric that is calculated as the geometric mean of the sensitivity and specificity metrics.
The example below demonstrates how to calculate the geometric mean for a list of 10 numbers.
1
2
3
4
5
6
7
# example of calculating the geometric mean
from scipy.stats import gmean
# define the dataset
data=[1,2,3,40,50,60,0.7,0.88,0.9,1000]
# calculate the mean
result=gmean(data)
print('Geometric Mean: %.3f'%result)
Running the example calculates the geometric mean and reports the result.
1
Geometric Mean: 7.246
Harmonic Mean
The harmonic mean is calculated as the number of values N divided by the sum of the reciprocal of the values (1 over each value).
Harmonic Mean = N / (1/x1 + 1/x2 + … + 1/xN)
If there are just two values (x1 and x2), a simplified calculation of the harmonic mean can be calculated as:
Harmonic Mean = (2 * x1 * x2) / (x1 + x2)
The harmonic mean is the appropriate mean if the data is comprised of rates.
Recall that a rate is the ratio between two quantities with different measures, e.g. speed, acceleration, frequency, etc.
In machine learning, we have rates when evaluating models, such as the true positive rate or the false positive rate in predictions.
The harmonic mean does not take rates with a negative or zero value, e.g. all rates must be positive.
One common example of the use of the harmonic mean in machine learning is in the calculation of the F-Measure (also the F1-Measure or the Fbeta-Measure); that is a model evaluation metric that is calculated as the harmonic mean of the precision and recall metrics.
Hello Jason,
does it exist any statistic about usage of Keras vs. Sklearn.
People prefer, up to you , using Keras or Sklearn? (because it is easier to use)
It seems to me there are several “overlapping ” functions between Keras and Sklearn to get the same result. What you think about?
Thanks
I have a genomic data with markers and their depth. Since the totals number of reads are always different sample to sample, we usually use normalization(divided by mean or calculate zscore)
However, I have a very sparse data. Some markers have high depth but most of markers have 1 or 2 depths or zero.
Here’s the example.
(0,0,0,0,0,0,1,1,1,1,1,1,1,2,2,2,2,10,20,30)
One sample has more than 2 million markers having similar pattern as above.
I wonder what kind of normalization will be suitable for my data.
It is the same unit but looks like different unit.
Why would anybody consider computing a mean of values having differing units? Sounds bad from physical point of view. Could you give some example or explanation?
Good example of application! Still specificity and sensitivity are dimensionless quantities so having same unit as far as I am concerned. I would be truly surprised by one trying to average e.g. meters and seconds!
hello dear Jason,
let me know the reasons of applying geaomeric mean instead of the other means?, thnks for your answer
ali asghar ghalavand , from Iran
12, aug..2022
I have to calculate the mean of uptime of machine in factory. Uptime is basically a dimensionless ratio (you can approximate it to uptime = no.of.units produced/Production plan)
But if i have to report 1 number(reason to calculate mean) for a machine for a given factory, what is appropriate measure – Arithmetic,Geometric or Harmonic and why ?
It would be really helpful if you can help me understand the reason specific to the above question
Nice article. Your readers may be interested in some studies I’ve completed showing how the geometric mean relates to Shannon entropy. For machine learning, the cross-entropy metric used to measure the accuracy of probabilistic inferences can be translated to a probability metric and becomes the geometric mean of the probabilities.
The arithmetic mean of probabilities filters out outliers – low probabilities – and as such can be used to measure how Decisive an algorithm is. This measurement is similar to the classification performance of the algorithm.
The harmonic mean of probabilities turns out to be too sensitive to outliers. But the -2/3rds mean is a mathematical conjugate of the arithmetic mean and gives good insight into the Robustness of algorithm.
Hi, I saw you have a book “Statistical Methods for Machine Learning” and others – would be great if they would be accessible as an AudioBook on Audible – was looking for it there but could not find it.
Thank you for the suggestion Shai! Our material is meant to be used to get you started with machine learning concepts and present code that can be immediately used to get results. For this reason, our current format works best keeping you engaged by actively running the code samples in your machine learning environment.
hi I have a question im not sure if you specialise in this but I am doing a histogram for my assignment and it has a unimodal normal distribution curve which means you should find the overall mean of the data arithmetically however the class intervals in my data set are uneven so it suggests I should find the mean of the intervals geometrically. Just asking whether you have to find the mean interval arithmetically since you are calculating the overall mean arithmetically or if you can find the mean of the intervals geometrically since they are uneven then calculate the overall mean arithmetically.
I have a question im not sure if you specialise in this but I am doing a histogram for my assignment and it has a unimodal normal distribution curve which means you should find the overall mean of the data arithmetically however the class intervals in my data set are uneven so it suggests I should find the mean of the intervals geometrically. Just asking whether you have to find the mean interval arithmetically since you are calculating the overall mean arithmetically or if you can find the mean of the intervals geometrically since they are uneven then calculate the overall mean arithmetically.
I am curious what you meant by “…if the data contains negative or zero values, then the geometric and harmonic means cannot be used directly.”
More specifically, I am working with data with negative rate values, so calculating the harmonic mean wouldn’t traditionally work as you highlighted above. However, I was wondering if you had in mind any indirect methods to achieve similar outcomes as calculating harmonic means for these values.
I am unclear about geometric mean, which “is appropriate if the values have differing units.” How would you have data measuring the same variable to have different units in the first place? The example given about sensitivity and specificity was confusing because those values have no units, they are probabilities.
Similarly, the harmonic mean “is appropriate if the data values are ratios of two variables with different measures, called rates.” Again, the example provided about recall and precision are also unitless, as they are also probabilities.
Hello Jason,
does it exist any statistic about usage of Keras vs. Sklearn.
People prefer, up to you , using Keras or Sklearn? (because it is easier to use)
It seems to me there are several “overlapping ” functions between Keras and Sklearn to get the same result. What you think about?
Thanks
sklearn is for machine learning.
Keras is for deep learning.
Hi Jason
I have a genomic data with markers and their depth. Since the totals number of reads are always different sample to sample, we usually use normalization(divided by mean or calculate zscore)
However, I have a very sparse data. Some markers have high depth but most of markers have 1 or 2 depths or zero.
Here’s the example.
(0,0,0,0,0,0,1,1,1,1,1,1,1,2,2,2,2,10,20,30)
One sample has more than 2 million markers having similar pattern as above.
I wonder what kind of normalization will be suitable for my data.
It is the same unit but looks like different unit.
I guess it depends why you want to scale the data?
Perhaps explore robust scaling, that is invariant to outliers?
Why would anybody consider computing a mean of values having differing units? Sounds bad from physical point of view. Could you give some example or explanation?
Regards!
It is used to average specificity and sensitivity for imbalanced classification:
https://machinelearningmastery.com/tour-of-evaluation-metrics-for-imbalanced-classification/
Good example of application! Still specificity and sensitivity are dimensionless quantities so having same unit as far as I am concerned. I would be truly surprised by one trying to average e.g. meters and seconds!
Regards!
Agreed.
hello dear Jason,
let me know the reasons of applying geaomeric mean instead of the other means?, thnks for your answer
ali asghar ghalavand , from Iran
12, aug..2022
Hi ghalavand…you may find the following resource of interest:
https://towardsdatascience.com/on-average-youre-using-the-wrong-average-geometric-harmonic-means-in-data-analysis-2a703e21ea0
Hi Jason,
I have to calculate the mean of uptime of machine in factory. Uptime is basically a dimensionless ratio (you can approximate it to uptime = no.of.units produced/Production plan)
But if i have to report 1 number(reason to calculate mean) for a machine for a given factory, what is appropriate measure – Arithmetic,Geometric or Harmonic and why ?
It would be really helpful if you can help me understand the reason specific to the above question
I could be wrong, but my intuition suggests a harmonic mean, e.g. given it is a ratio or rate.
Hello Jason,
Nice article. Your readers may be interested in some studies I’ve completed showing how the geometric mean relates to Shannon entropy. For machine learning, the cross-entropy metric used to measure the accuracy of probabilistic inferences can be translated to a probability metric and becomes the geometric mean of the probabilities.
The arithmetic mean of probabilities filters out outliers – low probabilities – and as such can be used to measure how Decisive an algorithm is. This measurement is similar to the classification performance of the algorithm.
The harmonic mean of probabilities turns out to be too sensitive to outliers. But the -2/3rds mean is a mathematical conjugate of the arithmetic mean and gives good insight into the Robustness of algorithm.
More details are provided in these two articles:
Assessing Probabilistic Inferences:
https://www.mdpi.com/1099-4300/19/6/286
Reduced Perplexity:
https://arxiv.org/abs/1603.08830
Thanks for sharing.
between arithmetic mean harmonic mean weighted mean and geometric which one of them doesnt depend on the total number of observation
Not sure I follow, I guess they all depend on the values of the samples not the number of samples.
Hi, I saw you have a book “Statistical Methods for Machine Learning” and others – would be great if they would be accessible as an AudioBook on Audible – was looking for it there but could not find it.
Thank you for the suggestion Shai! Our material is meant to be used to get you started with machine learning concepts and present code that can be immediately used to get results. For this reason, our current format works best keeping you engaged by actively running the code samples in your machine learning environment.
hi I have a question im not sure if you specialise in this but I am doing a histogram for my assignment and it has a unimodal normal distribution curve which means you should find the overall mean of the data arithmetically however the class intervals in my data set are uneven so it suggests I should find the mean of the intervals geometrically. Just asking whether you have to find the mean interval arithmetically since you are calculating the overall mean arithmetically or if you can find the mean of the intervals geometrically since they are uneven then calculate the overall mean arithmetically.
sorry about the long paragraph thank you!
hi jason
I have a question im not sure if you specialise in this but I am doing a histogram for my assignment and it has a unimodal normal distribution curve which means you should find the overall mean of the data arithmetically however the class intervals in my data set are uneven so it suggests I should find the mean of the intervals geometrically. Just asking whether you have to find the mean interval arithmetically since you are calculating the overall mean arithmetically or if you can find the mean of the intervals geometrically since they are uneven then calculate the overall mean arithmetically.
sorry about the long paragraph thank you!
Hi husna…I highly recommend investigating Seaborn for your purposes.
https://machinelearningmastery.com/seaborn-data-visualization-for-machine-learning/
Hi Jason,
I am curious what you meant by “…if the data contains negative or zero values, then the geometric and harmonic means cannot be used directly.”
More specifically, I am working with data with negative rate values, so calculating the harmonic mean wouldn’t traditionally work as you highlighted above. However, I was wondering if you had in mind any indirect methods to achieve similar outcomes as calculating harmonic means for these values.
Thank you!
Best,
Ryan
Hi Ryan…The following resource should add clarity:
https://www.toppr.com/guides/business-mathematics-and-statistics/measures-of-central-tendency-and-dispersion/harmonic-geometric-mean/
I am unclear about geometric mean, which “is appropriate if the values have differing units.” How would you have data measuring the same variable to have different units in the first place? The example given about sensitivity and specificity was confusing because those values have no units, they are probabilities.
Similarly, the harmonic mean “is appropriate if the data values are ratios of two variables with different measures, called rates.” Again, the example provided about recall and precision are also unitless, as they are also probabilities.
Hi Ming…The following resources may be of interest:
https://towardsdatascience.com/on-average-youre-using-the-wrong-average-geometric-harmonic-means-in-data-analysis-2a703e21ea0