Last Updated on

Statistics is a collection of tools that you can use to get answers to important questions about data.

You can use descriptive statistical methods to transform raw observations into information that you can understand and share. You can use inferential statistical methods to reason from small samples of data to whole domains.

In this post, you will discover clearly why statistics is important in general and for machine learning and generally the types of methods that are available.

After reading this post, you will know:

- Statistics is generally considered a prerequisite to the field of applied machine learning.
- We need statistics to help transform observations into information and to answer questions about samples of observations.
- Statistics is a collection of tools developed over hundreds of years for summarizing data and quantifying properties of a domain given a sample of observations.

Discover statistical hypothesis testing, resampling methods, estimation statistics and nonparametric methods in my new book, with 29 step-by-step tutorials and full source code.

Let’s get started.

## Statistics is Required Prerequisite

Machine learning and statistics are two tightly related fields of study. So much so that statisticians refer to machine learning as “*applied statistics*” or “*statistical learning*” rather than the computer-science-centric name.

Machine learning is almost universally presented to beginners assuming that the reader has some background in statistics. We can make this concrete with a few cherry picked examples.

Take a look at this quote from the beginning of a popular applied machine learning book titled “*Applied Predictive Modeling*“:

… the reader should have some knowledge of basic statistics, including variance, correlation, simple linear regression, and basic hypothesis testing (e.g. p-values and test statistics).

— Page vii, Applied Predictive Modeling, 2013

Here’s another example from the popular “*Introduction to Statistical Learning*” book:

We expect that the reader will have had at least one elementary course in statistics.

— Page 9, An Introduction to Statistical Learning with Applications in R, 2013.

Even when statistics is not a prerequisite, some primitive prior knowledge is required as can be seen in this quote from the widely read “*Programming Collective Intelligence*“:

… this book does not assume you have any prior knowledge of […] or statistics. […] but having some knowledge of trigonometry and basic statistics will help you understand the algorithms.

— Page xiii, Programming Collective Intelligence: Building Smart Web 2.0 Applications, 2007.

In order to be able to understand machine learning, some basic understanding of statistics is required.

To see why this is the case, we must first understand why we need the field of statistics in the first place.

### Need help with Statistics for Machine Learning?

Take my free 7-day email crash course now (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

## Why Learn Statistics?

Raw observations alone are data, but they are not information or knowledge.

Data raises questions, such as:

- What is the most common or expected observation?
- What are the limits on the observations?
- What does the data look like?

Although they appear simple, these questions must be answered in order to turn raw observations into information that we can use and share.

Beyond raw data, we may design experiments in order to collect observations. From these experimental results we may have more sophisticated questions, such as:

- What variables are most relevant?
- What is the difference in an outcome between two experiments?
- Are the differences real or the result of noise in the data?

Questions of this type are important. The results matter to the project, to stakeholders, and to effective decision making.

Statistical methods are required to find answers to the questions that we have about data.

We can see that in order to both understand the data used to train a machine learning model and to interpret the results of testing different machine learning models, that statistical methods are required.

This is just the tip of the iceberg as each step in a predictive modeling project will require the use of a statistical method.

## What is Statistics?

Statistics is a subfield of mathematics.

It refers to a collection of methods for working with data and using data to answer questions.

Statistics is the art of making numerical conjectures about puzzling questions. […] The methods were developed over several hundred years by people who were looking for answers to their questions.

— Page xiii, Statistics, Fourth Edition, 2007.

It is because the field is comprised of a grab bag of methods for working with data that it can seem large and amorphous to beginners. It can be hard to see the line between methods that belong to statistics and methods that belong to other fields of study. Often a technique can be both a classical method from statistics and a modern algorithm used for feature selection or modeling.

Although a working knowledge of statistics does not require deep theoretical knowledge, some important and easy-to-digest theorems from the relationship between statistics and probability can provide a valuable foundation.

Two examples include the law of large numbers and the central limit theorem; the first aids in understanding why bigger samples are often better and the second provides a foundation for how we can compare the expected values between samples (e.g mean values).

When it comes to the statistical tools that we use in practice, it can be helpful to divide the field of statistics into two large groups of methods: descriptive statistics for summarizing data and inferential statistics for drawing conclusions from samples of data.

Statistics allow researchers to collect information, or data, from a large number of people and then summarize their typical experience. […] Statistics are also used to reach conclusions about general differences between groups. […] Statistics can also be used to see if scores on two variables are related and to make predictions.

Pages ix-x, Statistics in Plain English, Third Edition, 2010.

### Descriptive Statistics

Descriptive statistics refer to methods for summarizing raw observations into information that we can understand and share.

Commonly, we think of descriptive statistics as the calculation of statistical values on samples of data in order to summarize properties of the sample of data, such as the common expected value (e.g. the mean or median) and the spread of the data (e.g. the variance or standard deviation).

Descriptive statistics may also cover graphical methods that can be used to visualize samples of data. Charts and graphics can provide a useful qualitative understanding of both the shape or distribution of observations as well as how variables may relate to each other.

### Inferential Statistics

Inferential statistics is a fancy name for methods that aid in quantifying properties of the domain or population from a smaller set of obtained observations called a sample.

Commonly, we think of inferential statistics as the estimation of quantities from the population distribution, such as the expected value or the amount of spread.

More sophisticated statistical inference tools can be used to quantify the likelihood of observing data samples given an assumption. These are often referred to as tools for statistical hypothesis testing, where the base assumption of a test is called the null hypothesis.

There are many examples of inferential statistical methods given the range of hypothesises we may assume and the constraints we may impose on the data in order to increase the power or likelihood that the finding of the test is correct.

## Further Reading

This section provides more resources on the topic if you are looking to go deeper.

### Books

- Applied Predictive Modeling, 2013
- An Introduction to Statistical Learning with Applications in R, 2013.
- Programming Collective Intelligence: Building Smart Web 2.0 Applications, 2007.
- Statistics, Fourth Edition, 2007.
- All of Statistics: A Concise Course in Statistical Inference, 2004.
- Statistics in Plain English, Third Edition, 2010.

### Articles

- Statistics on Wikipedia
- Portal:Statistics on Wikipedia
- List of statistics articles on Wikipedia
- Mathematical statistics on Wikipedia
- History of statistics on Wikipedia
- Descriptive Statistics on Wikipedia
- Statistical Inference on Wikipedia

## Summary

In this post, you discovered clearly why statistics is important in general and for machine learning, and generally the types of methods that are available.

Specifically, you learned:

- Statistics is generally considered a prerequisite to the field of applied machine learning.
- We need statistics to help transform observations into information and to answer questions about samples of observations.
- Statistics is a collection of tools developed over hundreds of years for summarizing data and quantifying properties of a domain given a sample of observations.

Do you have any questions?

Ask your questions in the comments below and I will do my best to answer.

Hi

If dataset is tall. Then how do we sample it? I mean what methods are used for sample selection.

Regards

By tall, I guess you mean many rows. You can randomly select rows as a sub-sample.

Yes i mean largw number of rows. But how may i get samples of good quality to represent data in majority. Which method to be used

Often descriptive statistics can be used to confirm that a data sample is representative of the population.

Hypothesis tests can confirm these findings.

I think Khan use Central Limit Theorem by taking different sample means of rows…

How can we collaborate these statistic skills with programming and apply them for solving the real world problems, most probably for machine learning and AI problems?

Good question, here are 10 examples:

https://machinelearningmastery.com/statistical-methods-in-an-applied-machine-learning-project/

Hi,

1) Is descriptive statistics and EDA are same?

2) How descriptive statistics used in applied machine learning?

3) How inferential statistics used in applied machine learning?

Thank you

EDA is a process that can use descriptive stats.

Descriptive stats can inform how to better prepare data for modeling, perhaps.

ML is applied inference. We are building inductive models.

What is normal distribution

How is it related to sample size and representative sample

This post will help:

https://machinelearningmastery.com/statistical-data-distributions/

Thank you.

Is it safe to say, a normal distribution shows a representative sample of the population?

No. A sample may or may not be normal and may or may not be representative.

I understand a sample may or may not be normal or representative but if it is normal, would that be representative.

Thank you.

I would not make that claim. It could be normal, but underpowered and therefore not representative.

do classifier depends on mode mean and median if yes then how and why, how these statistics help us in selection of classifier.

It depends on the algorithm. Some do.

if a dataset has four columns each column has its own mean value… how will we get just one mean for the whole dataset

You don’t.

What are you trying to achieve exactly?

if my data set is arranged in 4 columns …. all i want is an overall one mean value for the whole dataset not 4 mean values for four columns.

If all columns measure the same thing, then perhaps stack them into one column and calculate the mean.

If not, calculating the mean across columns is invalid and would not have any meaning.