Difference Between Classification and Regression in Machine Learning

There is an important difference between classification and regression problems.

Fundamentally, classification is about predicting a label and regression is about predicting a quantity.

I often see questions such as:

How do I calculate accuracy for my regression problem?

Questions like this are a symptom of not truly understanding the difference between classification and regression and what accuracy is trying to measure.

In this tutorial, you will discover the differences between classification and regression.

After completing this tutorial, you will know:

  • That predictive modeling is about the problem of learning a mapping function from inputs to outputs called function approximation.
  • That classification is the problem of predicting a discrete class label output for an example.
  • That regression is the problem of predicting a continuous quantity output for an example.

Let’s get started.

Difference Between Classification and Regression in Machine Learning

Difference Between Classification and Regression in Machine Learning
Photo by thomas wilson, some rights reserved.

Tutorial Overview

This tutorial is divided into 5 parts; they are:

  1. Function Approximation
  2. Classification
  3. Regression
  4. Classification vs Regression
  5. Converting Between Classification and Regression Problems

Function Approximation

Predictive modeling is the problem of developing a model using historical data to make a prediction on new data where we do not have the answer.

Predictive modeling can be described as the mathematical problem of approximating a mapping function (f) from input variables (X) to output variables (y). This is called the problem of function approximation.

The job of the modeling algorithm is to find the best mapping function we can given the time and resources available.

For more on approximating functions in applied machine learning, see the post:

Generally, we can divide all function approximation tasks into classification tasks and regression tasks.

Classification Predictive Modeling

Classification predictive modeling is the task of approximating a mapping function (f) from input variables (X) to discrete output variables (y).

The output variables are often called labels or categories. The mapping function predicts the class or category for a given observation.

For example, an email of text can be classified as belonging to one of two classes: “spam and “not spam“.

  • A classification problem requires that examples be classified into one of two or more classes.
  • A classification can have real-valued or discrete input variables.
  • A problem with two classes is often called a two-class or binary classification problem.
  • A problem with more than two classes is often called a multi-class classification problem.
  • A problem where an example is assigned multiple classes is called a multi-label classification problem.

It is common for classification models to predict a continuous value as the probability of a given example belonging to each output class. The probabilities can be interpreted as the likelihood or confidence of a given example belonging to each class. A predicted probability can be converted into a class value by selecting the class label that has the highest probability.

For example, a specific email of text may be assigned the probabilities of 0.1 as being “spam” and 0.9 as being “not spam”. We can convert these probabilities to a class label by selecting the “not spam” label as it has the highest predicted likelihood.

There are many ways to estimate the skill of a classification predictive model, but perhaps the most common is to calculate the classification accuracy.

The classification accuracy is the percentage of correctly classified examples out of all predictions made.

For example, if a classification predictive model made 5 predictions and 3 of them were correct and 2 of them were incorrect, then the classification accuracy of the model based on just these predictions would be:

An algorithm that is capable of learning a classification predictive model is called a classification algorithm.

Regression Predictive Modeling

Regression predictive modeling is the task of approximating a mapping function (f) from input variables (X) to a continuous output variable (y).

A continuous output variable is a real-value, such as an integer or floating point value. These are often quantities, such as amounts and sizes.

For example, a house may be predicted to sell for a specific dollar value, perhaps in the range of $100,000 to $200,000.

  • A regression problem requires the prediction of a quantity.
  • A regression can have real valued or discrete input variables.
  • A problem with multiple input variables is often called a multivariate regression problem.
  • A regression problem where input variables are ordered by time is called a time series forecasting problem.

Because a regression predictive model predicts a quantity, the skill of the model must be reported as an error in those predictions.

There are many ways to estimate the skill of a regression predictive model, but perhaps the most common is to calculate the root mean squared error, abbreviated by the acronym RMSE.

For example, if a regression predictive model made 2 predictions, one of 1.5 where the expected value is 1.0 and another of 3.3 and the expected value is 3.0, then the RMSE would be:

A benefit of RMSE is that the units of the error score are in the same units as the predicted value.

An algorithm that is capable of learning a regression predictive model is called a regression algorithm.

Some algorithms have the word “regression” in their name, such as linear regression and logistic regression, which can make things confusing because linear regression is a regression algorithm whereas logistic regression is a classification algorithm.

Classification vs Regression

Classification predictive modeling problems are different from regression predictive modeling problems.

  • Classification is the task of predicting a discrete class label.
  • Regression is the task of predicting a continuous quantity.

There is some overlap between the algorithms for classification and regression; for example:

  • A classification algorithm may predict a continuous value, but the continuous value is in the form of a probability for a class label.
  • A regression algorithm may predict a discrete value, but the discrete value in the form of an integer quantity.

Some algorithms can be used for both classification and regression with small modifications, such as decision trees and artificial neural networks. Some algorithms cannot, or cannot easily be used for both problem types, such as linear regression for regression predictive modeling and logistic regression for classification predictive modeling.

Importantly, the way that we evaluate classification and regression predictions varies and does not overlap, for example:

  • Classification predictions can be evaluated using accuracy, whereas regression predictions cannot.
  • Regression predictions can be evaluated using root mean squared error, whereas classification predictions cannot.

Convert Between Classification and Regression Problems

In some cases, it is possible to convert a regression problem to a classification problem. For example, the quantity to be predicted could be converted into discrete buckets.

For example, amounts in a continuous range between $0 and $100 could be converted into 2 buckets:

  • Class 0: $0 to $49
  • Class 1: $50 to $100

This is often called discretization and the resulting output variable is a classification where the labels have an ordered relationship (called ordinal).

In some cases, a classification problem can be converted to a regression problem. For example, a label can be converted into a continuous range.

Some algorithms do this already by predicting a probability for each class that in turn could be scaled to a specific range:

Alternately, class values can be ordered and mapped to a continuous range:

  • $0 to $49 for Class 1
  • $50 to $100 for Class 2

If the class labels in the classification problem do not have a natural ordinal relationship, the conversion from classification to regression may result in surprising or poor performance as the model may learn a false or non-existent mapping from inputs to the continuous output range.

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Summary

In this tutorial, you discovered the difference between classification and regression problems.

Specifically, you learned:

  • That predictive modeling is about the problem of learning a mapping function from inputs to outputs called function approximation.
  • That classification is the problem of predicting a discrete class label output for an example.
  • That regression is the problem of predicting a continuous quantity output for an example.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

240 Responses to Difference Between Classification and Regression in Machine Learning

  1. Avatar
    Rizwan ali December 11, 2017 at 7:14 am #

    nice post to clear the basic concepts

  2. Avatar
    shivaprasad December 11, 2017 at 6:47 pm #

    Really a good one sir,i had little bit confusion regarding these two.Thank you very much

    • Avatar
      Jason Brownlee December 12, 2017 at 5:27 am #

      I’m glad it helped.

    • Avatar
      Bilal September 9, 2021 at 9:12 pm #

      I’m excited with this, thanks

  3. Avatar
    Kishore December 11, 2017 at 8:54 pm #

    Very specific and crystal clear.. we need more of these in your books too 🙂

  4. Avatar
    James December 13, 2017 at 12:02 am #

    “Some algorithms have the word “regression” in their name, such as linear regression and logistic regression, which can make things confusing because linear regression is a regression algorithm whereas logistic regression is a classification algorithm”

    Thank you for that clarification, even after all this time in ML that always nagged at me.\

  5. Avatar
    Andrey Koch December 15, 2017 at 5:36 am #

    This topic is known to me yet the article made it straight to the point, thanks!

    One thing, in “A classification can have real-valued or discrete input variables.” you probably meant continious vs. discrete

  6. Avatar
    Derek Amah December 15, 2017 at 5:44 am #

    Very basic and insightful. I experience same misunderstanding from people when hiring talent and sometimes explaining these concepts to people. Will take a cue from your explanation going forward. Thanks !!!!

  7. Avatar
    Monsef December 15, 2017 at 6:12 am #

    Valuable tutorial Jason

    Thank you very much

  8. Avatar
    youssef December 15, 2017 at 8:25 am #

    Merci bien pour votre clarification de la différences entre les deux types ..

  9. Avatar
    Manika.rao December 15, 2017 at 4:15 pm #

    Thank a lot Sir.its indeed a good post for basics on ml….

  10. Avatar
    JP December 16, 2017 at 3:20 am #

    Hey Jason,

    This is really helpful for beginners.
    Could you provide single example of following section:
    “Convert Between Classification and Regression Problems”

  11. Avatar
    MJ December 18, 2017 at 7:13 am #

    Great explanation of important concepts. Liked your comment on logistic regresión being a classification problem. The regression in the name does make it confusing when starting out.

    • Avatar
      Jason Brownlee December 18, 2017 at 3:22 pm #

      I’m glad it helped!

    • Avatar
      Shivani Gupt July 27, 2020 at 8:06 am #

      Very very well explained!Thankyou so much.
      Please give us a good example of converting between Classification and Regression predictive models!

  12. Avatar
    Idrees December 18, 2017 at 5:32 pm #

    I’m actually getting to focus on ML, absolutely new to it. Thanks for your eye-opening and insightful explanations

  13. Avatar
    alejandro Camargo December 20, 2017 at 5:31 am #

    Thank you!

  14. Avatar
    Mark December 29, 2017 at 8:14 am #

    How is it possible to reverse the process of discretizing data given that there is a loss of information? For example, following the same example above values of $0 to $49 would be represented by categorical value of 0.

    Can’t figure out how the reverse is possible without knowing original value (in this case price in USD).

  15. Avatar
    Kamran January 1, 2018 at 8:45 am #

    Any help about CNN and RNN implementation using pima indian dataset in python

  16. Avatar
    JOHN JEFFRY MENDEZ January 22, 2018 at 8:01 pm #

    Great blog! could u please explain how to determine the variance and bias for a model prediction?

  17. Avatar
    doupanpan January 25, 2018 at 5:11 am #

    Thanks for the post, clearly explained. Also, I am thinking about using Python to try these classification. regression, models etc. is there a useful online tutorials to follow ?

  18. Avatar
    Oussama January 25, 2018 at 7:37 am #

    Nice explanation. thanks.

  19. Avatar
    Dhanashree February 1, 2018 at 9:08 pm #

    Thanks very well explained. !!

  20. Avatar
    Phil Mckay February 6, 2018 at 1:56 am #

    Hi Jason:

    I am a patent attorney, used to be a physics animal. I have written several hundred patent applications directed to software, mostly cyber security and cloud security system patents. Lately, like in every field, AI and ML in particular is everywhere. I am not afraid of the math, but the vocabulary was an issue – until I read your post.

    I can’t tell you how much I appreciated that. As you know, vocabulary, at least consistent vocabulary usage, is a bit of an issue in the data science and software worlds. Your post cleared up many questions in just a few minutes of reading – THANK YOU!

    You are an awesome writer and teacher

  21. Avatar
    Wafa February 9, 2018 at 4:04 am #

    Hi,

    Thank you for your great tutorials.
    Could you please give us a tutorial on how to classify images using transfer learning and Tensorflow?
    Or guide me where I can find great tutorial about it like yours?

  22. Avatar
    Ali Shan February 21, 2018 at 6:05 am #

    Appreciate your effort and clearly helpful.
    thanks !

  23. Avatar
    hans March 5, 2018 at 6:39 am #

    simple and valuable , thanks.

  24. Avatar
    KK March 15, 2018 at 6:06 pm #

    very well explained, actually the solution of my problem.
    “How do I calculate accuracy for my regression problem?” this was my actual question but now it’s clear.
    Thanks man !!

  25. Avatar
    Frederick Alfhendra March 30, 2018 at 2:58 am #

    Hello Sir, did you had any suggestion regarding the reference such as Journal or Book that I could use to explain about Regression Predictive Modeling?

    Thank you

  26. Avatar
    Frederick Alfhendra March 30, 2018 at 12:50 pm #

    The question will be, “why we use Regression Predictive Modeling for Stock index forecasting”, any suggestion will be really appreciated, thank you, Sir.

  27. Avatar
    Abhijeet April 1, 2018 at 4:57 pm #

    Thanks Jason for amazing article…you Rock Jason !!!

  28. Avatar
    disouja April 12, 2018 at 7:38 pm #

    please clear with example that what is called classification and what is regression problem. how to get to know the problem is a classification problem or a regression problem.

    • Avatar
      Jason Brownlee April 13, 2018 at 6:39 am #

      Classification is about predicting a label (e.g. ‘red’). Regression is about predicting a quantity (e.g. 100).

      Does that help?

  29. Avatar
    Julio Lee April 24, 2018 at 1:28 pm #

    Fantastic post, thank you for sharing! I was recently training a model as a binary classification problem using sigmoid as a single output. However, I was able to get far better results using MSE rather than binary cross-entropy. Since MSE is mostly used for regression, does this mean I was forced to convert it to a regression problem? This has been tickling my brain for quite some time now….

    • Avatar
      Jason Brownlee April 24, 2018 at 2:51 pm #

      Thanks.

      MSE with a sigmoid output function? Wow, and it worked without error?

      Be careful when evaluating the skill of the model, ensure it is doing what you think.

  30. Avatar
    Rupesh May 2, 2018 at 4:22 pm #

    Thanks………

  31. Avatar
    Danilo May 9, 2018 at 3:25 am #

    Excellent post! I appreciate too much your work Ph.D. Jason Brownlee.
    I have a question. What type of task should I perform if my dependent variable observations are dichotomous, but I need to infer continuous values?

    • Avatar
      Jason Brownlee May 9, 2018 at 6:27 am #

      If you have categorical inputs and require a real-valued output, this sounds like regression, and a challenging case.

      Perhaps you can try modeling it directly as a regression problem and see how you go. You may want to integer encode or one hot encode the inputs.

      • Avatar
        Danilo May 9, 2018 at 7:25 am #

        Thank you very much for your help!
        I have used two variants to determine my continuous output:
        1. The probability of belonging to the positive class in a classification model.
        2. The output of a regression model.
        The accuracy in terms of a CMC curve for the classification model outperformed the regression model. But I am not sure if I have misunderstood some results.
        Is it feasible to assume the probability of belonging to the positive class in a classification model as the similarity to this class? When the classifier outputs the probability (p) to belong to the negative class, I computed the probability to belong to the positive class as 1-p.

        • Avatar
          Jason Brownlee May 9, 2018 at 2:55 pm #

          Predicting class probability is a classification problem. Some algorithms can predict a probability.

          • Avatar
            Danilo May 9, 2018 at 3:17 pm #

            Thank you very much, Dr. Jason Brownlee! Your advice is helpful for my work. You are very gentle sharing your great knowledge!!!!

          • Avatar
            Jason Brownlee May 10, 2018 at 6:26 am #

            I’m glad it helped.

  32. Avatar
    Jefferson Sankara June 6, 2018 at 5:54 am #

    Dear Jason,

    The article really explained the concepts well. Thank you for the links to tutorials using Python.

  33. Avatar
    damak June 15, 2018 at 11:00 pm #

    This is simply awesome and it helped clarify a number of issues in ML. Thanks Dr. Brownlee

  34. Avatar
    Dhanushka Sanjaya June 16, 2018 at 12:49 am #

    thanks for this article. it is very helpfull

  35. Avatar
    alejo July 9, 2018 at 5:07 am #

    Hi, Jason.

    What happens when a problem has many ordinal categories? For example, if I wanted to predict, I don’t know, the goals (soccer) a team scores in a match (which generally will be in the range of 0 to 10) is a classification problem or a regression problem?

    • Avatar
      Jason Brownlee July 9, 2018 at 6:37 am #

      There are no rules. Try both and see which results in the better or more stable model.

  36. Avatar
    James Y July 18, 2018 at 4:48 am #

    A basic question: are the classifiers generally faster than the regressors because the discrete outputs are easier to compute?

  37. Avatar
    Shailendra July 19, 2018 at 2:55 pm #

    Thanks a lot for sharing this beautiful concept, i am looking for more.

  38. Avatar
    Vinit Tanna July 20, 2018 at 8:16 am #

    Really helpful.
    Now my concept is crystal clear!!
    Thanks 🙂

  39. Avatar
    william July 29, 2018 at 1:37 am #

    this is great, thanks again jason!

    is there a certain threshold of a RMSE for a regression problem that would provide the impetus to discretize the output variable into an ordinal classification problem? i.e. 0.5?

    i could be completely wrong here, but it seems like discretizing the output of a regression is simply just artificially inflating your predictive model’s precision accuracy…

    what are you thoughts?

    • Avatar
      Jason Brownlee July 29, 2018 at 6:13 am #

      I think it depends on the problem, whether the interpretation of the output as discrete quantities makes sense.

  40. Avatar
    Brandon August 11, 2018 at 1:28 am #

    Thanks

  41. Avatar
    Rawan September 13, 2018 at 2:54 pm #

    Thanks Jason! Your language in explaining the topic was easy to understand.

  42. Avatar
    Urise September 20, 2018 at 5:39 pm #

    Thanks Jason, it is great help to me.

    One thing I need to confirm, if I use movielens dataset to predict the rating, the regression prediction is the correct one, right? What if I use classification prediction in this case? May I got wrong prediction?

    • Avatar
      Jason Brownlee September 21, 2018 at 6:25 am #

      You can choose to model it as regression or classification.

      There is no objective “correct” in machine learning, there is just a dataset and your goals/stakeholders.

  43. Avatar
    Jani October 1, 2018 at 4:06 am #

    Are there any particular scenario that you can mention like a question which asks to implement Linear Regression and another question which ask to implement logistic regression (classification) so that we can implement in R.

    • Avatar
      Jason Brownlee October 1, 2018 at 6:31 am #

      When predicting a quantity, use linear regression.

      When predicting a binary label, use logistic regression.

  44. Avatar
    JG October 2, 2018 at 1:45 am #

    Hi jason,
    thanks.

    I consider this tutorial more convenient for my question about Regression vs Classification (classes).

    In the case you can convert your model from Regression to Classification, as it is explained in your last section, which could be more convenient to apply Regression (continuos) or Classification (discrete labels e.g. you divided your continuos output in some segment of interest and associated them to some artificial range such very low, low, medium, high, super, ultra, super ultra, tremendously, such it is the current case for frequency spectrum :-)))…Which model performs better Regression or Classification? using the same common layers (with the exception at the output layer and activation function?).

    • Avatar
      Jason Brownlee October 2, 2018 at 6:27 am #

      We cannot know which approach will perform best for a given problem, you must use systematic experimentation and discover what works best.

      This is the job of applied machine learning.

      • Avatar
        JG October 2, 2018 at 8:09 am #

        OK . Thks

  45. Avatar
    Data Scientist October 2, 2018 at 3:51 am #

    from content, you mentioned that A classification algorithm may predict a continuous value, but the continuous value is in the form of a probability for a class label.

    from the summary, you mentioned That classification is the problem of predicting a discrete class label output for an example.

    quite confusing

    • Avatar
      Jason Brownlee October 2, 2018 at 6:29 am #

      Sorry, perhaps ignore the prediction of a probability for now then and focus on predicting a real value for regression and a label for classification.

  46. Avatar
    Anuja Deokar October 9, 2018 at 2:47 am #

    When it is suitable to use regression over classification?

    • Avatar
      Jason Brownlee October 9, 2018 at 8:46 am #

      Use regression when you want to predict a quantity.

      Use classification when you want to predict a label.

      Does that help?

  47. Avatar
    Kotrappa October 16, 2018 at 9:47 pm #

    So nice discussion Thank you Jason , i am really interested in applications of Classification and Regression problems in real world like Automotive industry.

  48. Avatar
    Farhad October 28, 2018 at 7:13 pm #

    Hi Jason – I want the input to be the name of the event e.g. “Swimming” and the output to be like “Physical:80%, Social:10%, Emotional:10%”.

    Another Example:
    “Cooking” is “Vocational:30, Social:20, Nutritional:50”

    This is clearly not a classification problem, as the output is not binary. Do you agree it can be best implemented by regression models?

    Thanks,

    • Avatar
      Jason Brownlee October 29, 2018 at 5:55 am #

      It is a multi-label classification task.

      I don’t have examples of this type of task sorry.

  49. Avatar
    Boo October 30, 2018 at 10:48 am #

    Hi Jason. I was hoping if you could help clarify. You say to use regression for predicting quantities and classification for labels. If I was interested in the predicting the probability of something being in a specific class, would it still be a classification problem?

  50. Avatar
    Vipin Chauhan November 9, 2018 at 9:34 pm #

    You are always Awesome, Jason. Thanks for all your efforts. It cleared my doubt about these 2 ML methods.

  51. Avatar
    Praveena Chandra November 12, 2018 at 2:01 pm #

    Hi Jason,
    Thanks for the very informative blog. I do have a question. i want to predict Y, which can take the values 0,1or 2 based on an independent variable (X) which takes a value from 0 to 9. However, there are additional 8 categorical variables that are my control variables. I want to know if X predicts Y. I performed a ordinal logistic regression, however, I was told classification would be a better approach. My question is, how does one perform classification if control variables are to be taken into account too?

    • Avatar
      Jason Brownlee November 12, 2018 at 2:11 pm #

      Perhaps a model can take the dependent and control variables as input and predict y?

      Also, perhaps try a one hot encoding if the vars are ordinal/categorical.

  52. Avatar
    Max L. November 14, 2018 at 7:48 pm #

    Hi Jason,

    is it possible to generalize the amount of data that is needed for a regression and classification Problem?
    In my Opinion, to make a valid answer of how much something is, it needs more Data to learn from than “just” saying it is 1 or 0. Or is there no difference in the needed amount of Data for Regression/Classification Prolems?
    Thanks for an answer!

  53. Avatar
    Lolo December 19, 2018 at 3:37 am #

    Thank you for that great article. Back to how you start: “How do I calculate accuracy for my regression problem?” I am still not sure why this isn’t a valid question to ask. If your regression model predicts an amount between $0 and $100, we could still use various metrics to assess the accuracy of the prediction (average % error, max % error, error variance, etc.) No?

    • Avatar
      Jason Brownlee December 19, 2018 at 6:38 am #

      No. You calculate the error in the prediction for regression problems.

      You can transform the regression into a classification problem, predict labels (e.g. at $10 increments) and report accuracy of that, but, it may not be the best way to model the problem (e.g. harder).

  54. Avatar
    nikita December 19, 2018 at 6:34 am #

    This explanation is just too awesome man.I had already studied about these concepts in depth a month ago.But most of the concepts used to appear so difficult to me until I read ur post.I am about to start a ml project in a couple of days,and I can’t express how nicely ur post has cleared my concepts.Thank you so much

  55. Avatar
    Kp December 27, 2018 at 4:45 am #

    Brilliant explanation of confusing topics

  56. Avatar
    Abhishek December 29, 2018 at 12:26 am #

    why the output of the regression problem is called continuous?

    • Avatar
      Jason Brownlee December 29, 2018 at 5:53 am #

      It is a real valued number, e.g. fractional or no interruption – continuous.

    • Avatar
      Gull Mohammad October 9, 2019 at 5:50 am #

      Thanks a lot sir great explaination. How to convert one class classification problem to regression?

      • Avatar
        Jason Brownlee October 9, 2019 at 8:16 am #

        It depends on the problem, perhaps you can divide your class into ordinal values that make sense for the domain?

  57. Avatar
    Smriti January 11, 2019 at 1:14 am #

    Very clear explanation!!! Appreciate your efforts.

  58. Avatar
    Adelin January 24, 2019 at 8:10 pm #

    Really a great article ! Thank you very much ????

  59. Avatar
    Akanksha January 30, 2019 at 7:17 pm #

    This is a great article! I have read so many articles to understand this difference but none were so clear! Thanks!

  60. Avatar
    Aaron Dantley February 18, 2019 at 6:43 am #

    This was really helpful Jason! Thanks for the post.

  61. Avatar
    Mahmud February 19, 2019 at 11:34 pm #

    Your article is so wonderful
    Could you please guide or make an article about the other problems of ML like: clustering, Ranking, Associations and correlation

  62. Avatar
    Andana Anugrah February 23, 2019 at 9:17 pm #

    what should i do if want to make prediction that has output in discrete? for example i want to make prediction of the sum of product , and the input is behaviour of customer . can i predict using regression?

    • Avatar
      Jason Brownlee February 24, 2019 at 9:07 am #

      Yes, I don’t see why not.

      • Avatar
        Andana Anugrah February 24, 2019 at 11:06 pm #

        but if i make prediction with regression, the output will be continuous.
        let say the output is 67.3, in reality the product can’t be a fraction number.
        what should i do? ca i round the output?

        • Avatar
          Jason Brownlee February 25, 2019 at 6:42 am #

          Perhaps you can write some code to post-process the output?

          Perhaps you can find an alternate representation for the model output that correctly constrains the output values. e.g. discrete classes?

  63. Avatar
    sam McCall February 28, 2019 at 1:31 pm #

    Hello Doc! What does a two-class or a two-component regression data mean?

  64. Avatar
    Javad March 16, 2019 at 6:46 am #

    Hello,
    Thanks for your tutorials. What is the best strategy for a multi-class multi-label classification problem? For example, we have three outputs and each output has three classes.
    Thanks.

  65. Avatar
    jessy April 24, 2019 at 10:39 am #

    hi jason ,
    for classification problem -Is that forecasting possible ..

  66. Avatar
    Balaji June 1, 2019 at 8:08 pm #

    Thanks a bunch! This post was really helpful and concise.

  67. Avatar
    elle July 2, 2019 at 3:35 pm #

    Hi Jason, I am still confusing after reading here. How to distinguish between classification and regression method? My response may be categorical, but my purpose is estimating the probability of each categorical. Is this still a classification problem? Is the response type decided what the method is instead of the type of purpose? I am so confused.

  68. Avatar
    Jiawei Zhang July 4, 2019 at 2:47 pm #

    Hi Jason !

    Thank you so much for doing this tutorial.
    Could you show some example of how do we accomplish “range between $0 and $100 could be converted into 2 buckets:” in the actual code ?

    I completed a LSTM regression model right now and I want to convert it into classification model. How do i do that exactly ?

    • Avatar
      Jason Brownlee July 4, 2019 at 2:51 pm #

      Sorry, I cannot prepare code for you.

      • Avatar
        Jiawei Zhang July 4, 2019 at 3:14 pm #

        Got it. If the output that I want to predict is -1, 0, 1. What is the best approach using LSTM classification ?

        Thank you so much for answering

        • Avatar
          Jason Brownlee July 4, 2019 at 3:18 pm #

          LSTM is only appropriate if the input is a sequence.

          In that case, you can use a Dense output layer with a linear or perhaps a tanh activation function and fit with MSE.

  69. Avatar
    Vish July 27, 2019 at 1:55 am #

    Convert Between Classification and Regression Problems. I am doing a similar approach and would like to know more in detail about it.

    Where can I find it?

    • Avatar
      Jason Brownlee July 27, 2019 at 6:11 am #

      It is specific to the data.

      Perhaps just try exploring a few different framings of the problem and see what works well?

  70. Avatar
    Francisco August 19, 2019 at 11:48 am #

    Many thanks for yous post!!!!!

  71. Avatar
    Udaya Rayapur (Udax) August 26, 2019 at 1:00 am #

    Thanks Jason for valuable info and the effort you put in to share this.

  72. Avatar
    Nick September 23, 2019 at 7:07 am #

    Hi Jason, I feel a bit confused. By your definition, logistic regression is not regression but classification, is that correct?

    • Avatar
      Jason Brownlee September 23, 2019 at 8:53 am #

      Correct!

      It is a classification algorithm with a TERRIBLE name 🙂

      This is a common point of confusion, you’re not alone.

  73. Avatar
    Manish Rai November 1, 2019 at 3:29 pm #

    Nice explanation

  74. Avatar
    Jenny November 10, 2019 at 9:14 pm #

    I would say I’m really not good at math, but the way you can describe and teach complex things like this makes it even understandable for normal folks like me.

    THANK YOU !!

  75. Avatar
    Murilo Souza November 13, 2019 at 10:32 pm #

    Hello! I’ve been reading your tutorials last few days, and i’ve learned so much! Thanks!

    I have a question related to defining an activation function to my classification problem. Say i have 2 outputs in it, and one can have 3 classes and the other can have 2 classes, something like this:

    Output 1:

    Class 0: $0 to $50
    Class 1: $51 to $ 100
    Class 2: Greater than $100

    Output 2:

    Class 0: Less than $2000
    Class 1: Greater than $2000

    Can i just label (sorry, don’t know if this is the correct term) those classes as -1, 0, 1 for the first output, and -1, 1(Output 2) and use a hyperbolic tanget as activation function?

  76. Avatar
    George Zhou November 20, 2019 at 4:56 am #

    I hear people say “how accurate is the model” in reference to a regression problem all the time. This is how people were accustomed to saying it in grade school. How should I correct them? Instead of saying “how accurate is the model” what should we say?

    • Avatar
      Jason Brownlee November 20, 2019 at 6:23 am #

      What is the error of your model?
      What is the prediction error of your model?

  77. Avatar
    Herve November 22, 2019 at 6:24 pm #

    Very clear and concise, thanks !

  78. Avatar
    Harish Varma Alluri December 14, 2019 at 3:45 am #

    sir, I am working on seed classification and quality assessment using machine learning. what is the metric used for quality assessment? is it regression or classification.

    • Avatar
      Jason Brownlee December 14, 2019 at 6:24 am #

      If you are predicting a number, it is a regression and you calculate error in predictions.

      If you are predicting a label, it is classification and you calculate accuracy in predictions.

  79. Avatar
    Shiva Golla January 31, 2020 at 2:35 pm #

    Hi Jason, This one helped a lot but i have some doubts.

    1) I choose AirBnb data and want to predict the price for a given criteria by user. I mean i build a model and give my example data to predict price and it will give me a certain price. This is called Regression right?

    2) For the above problem, how should i know that how good is my model? (like method of calculation)

    Please help me out.
    Thank you in advance.

  80. Avatar
    Najeh February 8, 2020 at 8:48 pm #

    Hi Jason, thank you for this incredible tutorial.
    I’m working to use multifractal analysis for the classification process, did u have any tutorial about that please?

  81. Avatar
    G. Hulli February 22, 2020 at 2:44 am #

    Thank you Jason for the clear explanation.

    There are a lot of literature about this distinction online but none of them summarizes as clear and simple as you did here.

    Great stuff
    cheers!

  82. Avatar
    David February 27, 2020 at 9:33 am #

    Thank you so much for your articles It makes me more understand than before. By the way, I have a small question related to the phrases you wrote as i am a new learner for machine learning. The phrase:
    A problem with more than two classes is often called a multi-class classification problem.
    A problem where an example is assigned multiple classes is called a multi-label classification problem.
    I misunderstand between these two phrases. Can you give me small examples btw these two. Thank you much.

    • Avatar
      Jason Brownlee February 27, 2020 at 1:32 pm #

      Thanks.

      Yes.

      Multi-class classification has n classes that are mutually exclusive, e.g. red or blue.

      Multi-label classification has n classes that are mutually non-exclusive, e.g. red and/or blue or neither.

  83. Avatar
    Fayaz Ahmad March 12, 2020 at 9:13 pm #

    Thank you Jason for the clear explanation.

  84. Avatar
    Karon June 12, 2020 at 11:53 am #

    Is it possible to decompose a problem with a combination of regression + classification? For example, a dataset has two measures, measure 1 is continuous, measure 2 is binary. Using regression for measures, and classification for measure 2?

    • Avatar
      Jason Brownlee June 12, 2020 at 1:38 pm #

      Classification and regression refer to the target that is being predict, and typically it is one variable. You may have one or more input variables that are labels or numbers, but it not not impact whether the prediction problem is regression or classification.

      Does that help?

  85. Avatar
    Joachim July 1, 2020 at 11:56 am #

    What is the difference between linear regression and logistic regression? Is it correct that logistic regression uses the exact same formula as linear regression after it has raised all training data to an exponent and then normalized all the training data?

    • Avatar
      Jason Brownlee July 1, 2020 at 1:23 pm #

      Linear regression is for predicting a numerical value.

      Logistic regression is for predicting a binary class label.

  86. Avatar
    Heba July 4, 2020 at 9:24 am #

    Thank you for this useful clarification.

    I have a concern regarding explainable artificial intelligence (XAI). Based on literature, XAI is usually applied to classification algorithms, not regression algorithms. Is there a convincing reason for that?

    Thanks…

    • Avatar
      Jason Brownlee July 5, 2020 at 6:51 am #

      Sorry, I am not an expert in explainable AI, I hope to write about the topic in the future.

  87. Avatar
    Alan July 9, 2020 at 8:20 pm #

    How is logistical regression different to classification?

    If logistical regression give 0 or 1, y/n, is that different to classification?

    • Avatar
      Jason Brownlee July 10, 2020 at 5:57 am #

      Logistic regression is a classification algorithm.

  88. Avatar
    Isa Bashir Salisu July 14, 2020 at 9:54 pm #

    Very informative!!!
    Thanks a lot for the clarification.

  89. Avatar
    Kumar Amit August 24, 2020 at 11:10 pm #

    Hi Jason,

    Is it possible to do few-shot learning with SGD classifier?

    My requirement is to train my model with a NEW discrete class label incrementally (partial_fit) but without prior knowledge of NEW class in the first partial_fit training as it happened in SGD classifier with partial_fit.

    • Avatar
      Jason Brownlee August 25, 2020 at 6:42 am #

      I expect it depends on the specific dataset.

      Perhaps try it and see.

  90. Avatar
    Atta August 27, 2020 at 4:51 am #

    Hi Jason,

    Great post, very informative especially for people who just use it as a tool.

    Could you provide some references (books, paper,..etc)? so we can buy books or cite you for that information.

  91. Avatar
    yanamandra sai sarvana September 5, 2020 at 4:09 pm #

    very nice post

  92. Avatar
    Dominic Stalder September 7, 2020 at 2:57 pm #

    clear and straightforward explanation, thanks!

  93. Avatar
    Jana September 14, 2020 at 11:09 am #

    Everything posted made a ton of sense. However, consider this, what if you
    were to create a awesome headline? I am not saying your information is not good., but what if you added
    something that grabbed folk’s attention? I mean Difference Between Classification and Regression in Machine Learning is a little boring.
    You could glance at Yahoo’s front page and see how they create post
    headlines to grab viewers interested. You might try
    adding a video or a picture or two to grab people excited
    about what you’ve written. Just my opinion, it might make your posts
    a little livelier.

  94. Avatar
    Dr S P Sreenivas Padala November 4, 2020 at 8:08 pm #

    Awesome post. Not only this article but all articles in this blog. This blog is Wikipedia of Machine learning.

    Do all these posts are summarized anywhere in the form of a table of contents or in form of the mindmap. I want to connect all these posts in a sequence. It will help us to understand ML better i feel than randomly going to each post.

  95. Avatar
    Ammar Sani MY December 4, 2020 at 4:55 am #

    I once came here for understanding classification vs regression. I think I should understand these concepts since Simple Logistic in WEKA is the best classifier among others for my classification problem.

    However, seem this classifier is actually using a logistic regression behind it, I’m becoming confusing by the word ‘regression’ part of its name. Thinking and finding answer either I am doing correct actions to determine the best classifier of mine.

    Fortunately, just reading this post solve my entire problem.

    Thanks sir!

  96. Avatar
    Dinesh Durgadas Darp January 14, 2021 at 5:14 am #

    Very nice and understandable article. i enjoyed this page and have bookmarked it. This article has classified my understanding from 0 to 1(+ve)..

  97. Avatar
    Noel February 1, 2021 at 3:27 pm #

    Brilliantly explained for a 10-year-old. I really needed this.

  98. Avatar
    leonardo February 3, 2021 at 10:36 am #

    Hi Jason,
    many many thanks! this (and many others) were quite helpful for my regression problem using machine learning!

  99. Avatar
    Michael Owusu-Adjei April 16, 2021 at 6:03 pm #

    Great tutorial

  100. Avatar
    mo May 4, 2021 at 12:34 am #

    Logistic regression is actually classification problem. Love that they mentioned it explicitly

    • Avatar
      Jason Brownlee May 4, 2021 at 6:46 am #

      Yes, logistic regression is used for binary classification. Worst name ever.

  101. Avatar
    tilkesh May 10, 2021 at 8:32 pm #

    Is the image to image transformation using U-net is regression problem?

  102. Avatar
    Celiwe Kgowedi July 11, 2021 at 6:25 am #

    Hi
    Thank you so much for the tutorial. it helped me a lot in gaining more insight for my master’s research

  103. Avatar
    Alfredo Soto July 25, 2021 at 12:36 am #

    Very clear explanation of those concepts.
    Muchas Gracias

  104. Avatar
    MAKAN February 7, 2022 at 2:48 am #

    Hi Jason, I really enjoy your teaching but i have a question. when i use a basic neural network to model a non linear regression, i get the better result than using classification or clustering. which one of my steps do you think have problem. i use a simple neural network with two layers(input and output). but for example with decision trees and … i get bad result. Thank you in Advanced 🙂 .

    • Avatar
      James Carmichael February 7, 2022 at 12:07 pm #

      Hi Makan…Thank you for your feedback! You hit on a key consideration…that there are times when even “simple” networks can in fact perform better than more complicated networks for a given application. It is recommended therefore that the most simple model be selected that meets the required performance criteria.

  105. Avatar
    Elif A February 21, 2022 at 5:10 am #

    Thank you so much for clarification, it’ll help a lot on my project????

    • Avatar
      James Carmichael February 21, 2022 at 9:23 am #

      You are very welcome Elif A!

  106. Avatar
    Christossy March 5, 2022 at 6:50 am #

    Dear Jason
    Greetings,
    Sorry, my old computer was broken with all my data and even the books that I have purchased on 10th February 2022 were lost with a computer and I can not retrieve it. So, Kindly, I would like to get access to download the books again, please I need your help. The books are very important for my studies and career future. Find the forwarded

    I hope you will consider my request.

  107. Avatar
    shalini September 4, 2022 at 11:43 pm #

    Some algorithms cannot, or cannot easily be used for both problem types, such as linear regression for regression predictive modeling and logistic regression for classification predictive modeling.? this is supposed to be other way around, right?

    • Avatar
      James Carmichael September 5, 2022 at 8:44 am #

      Hi Shalini…Your understanding it correct!

  108. Avatar
    Adrian Olszewski November 17, 2023 at 6:03 pm #

    Logistic regression “is not a regression” only in Machine Learning. Elsewhere, in classic statistical analysis, for example in experimental research (like clinical trials), it’s used mostly for regression and inference purposes:
    – to assess the direction, magnitude and statistical inference about the parameters
    – to analyze (direction, magnitude, inference) the marginal effects
    – inference about the main and interaction effects (ANOVA, ANCOVA; here called analysis of deviance)
    – inference about the concrete comparisons (simple effects) via contrasts; both planned and post-hoc assessments
    – to extend a dozen classic tests of proportions and odds-ratios and stochastic superiority (z / chi2 test, Cochrane-Mantel-Haenszel, Cochrane-Armitage, Breslow-Day, McNemar / Cochran Q / Friedman, Mann-Whitney (Wilcoxon))
    – to implement the inverse probability weighting
    – to implement the propensity score matching.

    I use logistic regression on daily basis at work and have never used it for classification. We should distinguish the logistic CLASSIFIER (predicting classes) from the logistic REGRESSION (predicting – as every other regression – function of the conditional distribution of the response; here E(Y|X=x), which is of course numeric). It was invented (Cox, Nelder, Weddeburn, Hastie, Tibshirani) for solving regression purposes first, next to the probit regression and put into the frames of the GLM (Generalized Linear Model, covering linear, logistic, Poisson, gamma, beta and more regressions). Classification is only one of its applications in statistics.

    • Avatar
      James Carmichael November 18, 2023 at 10:18 am #

      Thank you for your input Adrian!

Leave a Reply