How to Think About Machine Learning

Machine learning is a large and interdisciplinary field of study.

You can achieve impressive results with machine learning and find solutions to very challenging problems. But this is only a small corner of the broader field of machine learning often called predictive modeling or predictive analytics.

In this post, you will discover how to change the way you think about machine learning in order to best serve you as a machine learning practitioner.

After reading this post, you will know:

  • What machine learning is and how it relates to artificial intelligence and statistics.
  • The corner of machine learning that you should focus on.
  • How to think about your problem and the machine learning solution to your problem.

Let’s get started.

How to Think About Machine Learning

How to Think About Machine Learning
Photo by Rajarshi MITRA, some rights reserved.

Overview

This post is divided into 3 parts; they are:

  1. You’re Confused
  2. What is Machine Learning?
  3. Your Machine Learning

You’re Confused

You have a machine learning problem to solve, but you’re confused about what exactly machine learning is.

There’s good reason to be confused. It is confusing to beginners.

Machine learning is a large field of study, and not all much of it is going to be relevant to you if you’re focused on solving a problem.

In this post, I hope to clear things up for you.

We will start off by describing machine learning in the broadest terms and how it relates to other fields of study like statistics and artificial intelligence.

After that, we will zoom in on the aspects of machine learning that you really need to know about for practical engineering and problem solving.

What is Machine Learning?

Machine learning is a field of computer science concerned with programs that learn.

The field of machine learning is concerned with the question of how to construct computer programs that automatically improve with experience.

Machine Learning, 1997.

That is super broad.

There are many types of learning, many types of feedback to learn from, and many things that can be learned.

This could encompass diverse types of learning, such as:

  • Developing code to investigate how populations of organisms “learn” to adapt to their environment over evolutionary time.
  • Developing code to investigate how one neuron in the brain “learns” in response to stimulus from other neurons.
  • Developing code to investigate how ants “learn” the optimal path from their home to their food source.

I give these esoteric examples on purpose to help you really nail down that machine learning is a broad and far reaching program of research.

Another case that you may be more familiar with is:

  • Developing code to investigate how to “learn” patterns in historical data.

This is less glamorous, but is the basis of the small corner of machine learning in which we as practitioners are deeply interested.

This corner is not distinct from the other examples; there can be a lot of overlap in methods for learning, fundamental tasks, ways of evaluating learning, and so on.

What About Artificial Intelligence?

Machine learning is a subfield of artificial intelligence.

It overlaps with machine learning.

Artificial intelligence is also an area of computer science, but it is concerned with developing programs that are intelligent, or can do intelligent things.

Intelligence involves learning, e.g. machine learning, but may involve other concerns such as reasoning, planning, memory, and much more.

This could encompass diverse types of learning such as:

  • Developing code to investigate how to optimally plan logistics.
  • Developing code to investigate how to reason about a paragraph of text.
  • Developing code to investigate how to perceive the contents of a photograph.

Artificial intelligence is often framed in the context of an agent in an environment with the intent to address some problem, but this does not have to be the case.

Machine learning could just as easily be named artificial learning to remain consistent with artificial intelligence and help out beginners.

The lines are blurry. Machine learning problems are also artificial intelligence problems.

What About Statistics?

Statistics, or applied statistics with computers, is a sub-field of mathematics that is concerned with describing and understanding the relationships in data.

This could encompass diverse types of learning such as:

  • Developing models to summarize the distribution of a variable.
  • Developing models to best characterize the relationship between two variables.
  • Developing models to test the similarity between two populations of observations.

It also overlaps with the corner of machine learning interested in learning patterns in data.

Many methods used for understanding data in statistics can be used in machine learning to learn patterns in data. These tasks could be called machine learning or applied statistics.

Your Machine Learning

Machine learning is a large field of study, and it can help you solve specific problems.

But you don’t need to know about all of it.

  • You’re not an academic investigating an esoteric type of learning as in machine learning.
  • You’re not trying to make an intelligent agent as in artificial intelligence.
  • You’re not interested in learning more about why variables relate to each other in data as in statistics.

In fact, when it comes to learning relationships in data:

  • You’re not investigating the capabilities of an algorithm.
  • You’re not developing an entirely new theory or algorithm.
  • You’re not extending an existing machine learning algorithm to new cases.

These may be activities in the corner of machine learning that we may be interested in, but activities for academics, not practitioners like you.

So what parts of machine learning do you need to focus on?

I think there are two ways to think about machine learning:

  1. In terms of the problem you are trying to solve.
  2. In terms of the solution you require.

Your Machine Learning Problem

Your problem can best be described as the following:

Find a model or procedure that makes best use of historical data comprised of inputs and outputs in order to skillfully predict outputs given new and unseen inputs in the future.

This is super specific.

First of all, it discards entire sub-fields of machine learning, such as unsupervised learning, to focus on one type of learning called supervised learning and all the algorithms that fit into that bucket.

That does not mean that you cannot leverage unsupervised methods; it just means that you do not focus your attention there, at least not to begin with.

Second of all, it gives you a clear objective that dominates all others: that is model skill at the expense of other concerns such as model complexity, model interpretability, and so on.

Again, this does not mean that these are not important, just that they are considered after or in conjunction with model skill.

Thirdly, the framing of your problem this way fits neatly into another field of study called predictive modeling. That is a field of study that borrows methods from machine learning with the objective of developing models that make skillful predictions.

In some areas of business, this area may also be called predictive analytics and encompasses more than just the modeling component to include related activities of gathering and preparing data and deploying and maintaining the model.

More recently, this activity can also be called data science, although that phrase also has connotations of inventing or discovering the problem in addition to working it through to a solution.

I don’t think it matters what you call this activity. But I do think it is important to deeply understand that your interest in and use of machine learning is highly specific and different from some other uses by academics.

It allows you to filter the material you read and the tools you choose in order to stay focused on the problem you’re trying to solve.

Your Machine Learning Solution

The solution you require is best described as the following:

A model or procedure that automatically creates the most likely approximation of the unknown underlying relationship between inputs and associated outputs in historical data.

Again, this is super specific.

You need an automatic method that produces a program or model that you can use to make predictions.

You cannot sit down and write code to solve your problem. It is entirely data-specific and you have a lot of data.

In fact, problems of this type resist top-down hand-coded solutions. If you could sit down and write some if-statements to solve your problem, you would not need a machine learning solution. It would be a programming problem.

The type of machine learning methods that you need will learn the relationship between the inputs and outputs in your historical data.

This framing allows you to think about what that real underlying yet unknown mapping function might look like and how noise, corruption, and sampling of your historical data may impact approximations of this mapping made by different modeling methods.

Without this framing, you will wonder things like:

  • Why there isn’t just one super algorithm or set of parameters.
  • Why the experts can’t just tell you what algorithm to use.
  • Why you can’t achieve a zero error rate with predictions from your model.

It helps you see the ill-defined nature of the predictive modeling problem you’re trying to solve and sets reasonable expectations.

Next Step

Now that you know how to think about machine learning, the next step is to change the way you think about the process of solving a problem with a machine learning solution.

For a hint, see the post:

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Posts

Articles

Summary

In this post, you discovered how to change the way you think about machine learning in order to best serve you as a machine learning practitioner.

Specifically, you learned:

  • What machine learning is and how it relates to artificial intelligence and statistics.
  • The corner of machine learning that you should focus on.
  • How to think about your problem and the machine learning solution to your problem.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

22 Responses to How to Think About Machine Learning

  1. Bart April 2, 2018 at 6:51 am #

    Hello Jason,

    Actually, I have somewhat opposite experience to what you wrote:
    “You cannot sit down and write code to solve your problem. It is entirely data-specific and you have a lot of data.”

    In fact, the access to data is an issue from my point of view. Do we need another algorithm or another proverbial neural network layer?

    In fact, my feeling is that we are armed with a lot on a programming side. However difficult / scarce access to data is a bottleneck (as opposed to facebooks / googles / amazons of this world)

    I wanted to ask if you have a hint/advice regarding the data access/collections for machine learning problems. What advice/tricks can you share?

    It seems like the dice are loaded, and small organisations (not to mention individuals) cannot compete with the evilcorps stated above

    • Jason Brownlee April 2, 2018 at 2:46 pm #

      Interesting perspective Bart.

      Perhaps get creative? Some ideas:

      – Perhaps offer free services to collect your own dataset?
      – Perhaps use data augmentation?
      – Perhaps use processes/simulations to generate synthetic data?

      I hope that stimulates some thoughts.

    • David Lowe April 5, 2018 at 5:31 am #

      Bart, For practicing machine learning, there are a number of data sources around just waiting to be tapped. Kaggle is one immediate example that comes to mind. There are many other sources that may be worthy of your consideration. I keep a list handy on my own website (https://daines-analytics.com/data-sources/) so I can look them up from time to time.

  2. Mahesh April 3, 2018 at 5:37 pm #

    Thanks for sharing, nice post for the practicle experience in the machine Learning visit Aretove

  3. babita April 4, 2018 at 9:39 pm #

    Thanks for sharing this amazing post on machine learning

  4. Shravan April 13, 2018 at 1:45 pm #

    Being focused is half problem solved!

  5. Kotrappa Sirbi April 15, 2018 at 1:03 am #

    Hello Sir,

    Nice article and explained in simple words to make understand what is machine learning, AI ,stats and Data Science. But sir, no where you mentioned importance of mathematics as basis for Machine Learning . Thanks

    • Jason Brownlee April 15, 2018 at 6:26 am #

      The math can come later. I recommend start by learning how to work through problems end to end, then circle back to the math later in order to better understand what you’re doing and how to get better results.

  6. JonS April 26, 2018 at 8:43 am #

    Hi, nice post. Do you have any recommended reading for if your focus IS designing an intelligent agent? My industry is mostly focused on ML algorithms for automated bidding. I’d love to read about some general frameworks for thinking about univariate TS models on sparse/volatile data

  7. Arshdeep Singh May 14, 2018 at 3:47 am #

    After learning machine learning concepts, algorithms related to it (SVM, logistic regression, ANN, naive Bayes, random forests, k means, etc) and a basic statistics(descriptive and inferential), what method or approach will you suggest a student to learn in order to solve the problems in society. What should I learn next? I would love to master this. Please advise. Thank you!

  8. Sam July 16, 2018 at 4:35 pm #

    Hi Jason Brownlee, This article is really helpful. I am looking for these type of articles. It cleared my doubt about Machine Learning.
    Good Article.
    Thanks, Jason Brownlee.

  9. shri September 24, 2018 at 11:02 am #

    Hi Jason,

    Thank you for developing and maintaining this website.

    I loved these two definitions:

    “Find a model or procedure that makes the best use of historical data comprised of inputs and outputs in order to skillfully predict outputs given new and unseen inputs in the future.”

    “A model or procedure that automatically creates the most likely approximation of the unknown underlying relationship between inputs and associated outputs in historical data.”

    I always thought that predictive analytics or developing a model should not require an in-depth programming knowledge nor should it require one to be a statistics guru, especially when it comes to solving people-related problems. (I may be wrong)

    I am a total newbie and always been confused about Statistics & ML and where does programming knowledge fit in.

    I am in the People Analytics field, understand a bit of statistics and want to use predictive analytics / modeling / ML to predict employee behavior.

    I thought of learning R/ Python and Statistics, but this bottom-up approach seems a bit daunting for me.

    Your website has given me some hope.

    What do you think is the best way forward for me?

    Thanks.

    Shri

    I have no programming background, do you think I can still What do you think is the best way forward for me?

  10. JG November 7, 2018 at 4:16 am #

    Hi Jason,

    Interesting reflections !

    In my opinion the big question in ML/DL, also it is the question that a company need to be addressed to the expert ML software developers is, what kinds of predictive questions can we ask to a (big) DATASET?.
    Latter on, technical issued like framing o setting up this (historical dataset) in terms of features, time, pixels, time-steps, signals, etc. beside the labels associated to each case of subset of the datasets, is coming next. In addition to define the right ANN models in terms of units, layers, loss, metrics, learning rate, epochs, batch, plus compile it… It is not easy to see if the question we can make can be managed (or not) by the ML/DL algorithms and the dataset build-up for each occasion.

    This the big challenge for me. Learn to ask questions to a ML process or defining the ML problem !

Leave a Reply