Non-Linear Regression in R

Last Updated on

In this post you will discover 4 recipes for non-linear regression in R.

There are many advanced methods you can use for non-linear regression, and these recipes are but a sample of the methods you could use.

Discover how to prepare data, fit machine learning models and evaluate their predictions in R with my new book, including 14 step-by-step tutorials, 3 projects, and full source code.

Let’s get started.

non-linear regression

Non-Linear Regression
Photo by Steve Jurvetson, some rights reserved

Each example in this post uses the longley dataset provided in the datasets package that comes with R. The longley dataset describes 7 economic variables observed from 1947 to 1962 used to predict the number of people employed yearly.

Multivariate Adaptive Regression Splines

Multivariate Adaptive Regression Splines (MARS) is a non-parametric regression method that models multiple nonlinearities in data using hinge functions (functions with a kink in them).

Learn more about the earth function and the earth package.

Support Vector Machine

Support Vector Machines (SVM) are a class of methods, developed originally for classification, that find support points that best separate classes. SVM for regression is called Support Vector Regression (SVM).

Learn more about the ksvm function and the kernlab package.

Need more Help with R for Machine Learning?

Take my free 14-day email course and discover how to use R on your project (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

Start Your FREE Mini-Course Now!

k-Nearest Neighbor

The k-Nearest Neighbor (kNN) does not create a model, instead it creates predictions from close data on-demand when a prediction is required. A similarity measure (such as Euclidean distance) is used to locate close data in order to make predictions.

Learn more about the knnreg function and the caret package.

Neural Network

A Neural Network (NN) is a graph of computational units that recieve inputs and transfer the result into an output that is passed on. The units are ordered into layers to connect the features of an input vector to the features of an output vector. With training, such as the Back-Propagation algorithm, neural networks can be designed and trained to model the underlying relationship in data.

Learn more about the nnet function and the nnet package.

Summary

In this post you discovered 4 non-linear regression methods with recipes that you can copy-and-paste for your own problems.

For more information see Chapter 7 of Applied Predictive Modeling by Kuhn and Johnson that provides an excellent introduction to non-linear regression with R for beginners.

Discover Faster Machine Learning in R!

Master Machine Learning With R

Develop Your Own Models in Minutes

...with just a few lines of R code

Discover how in my new Ebook:
Machine Learning Mastery With R

Covers self-study tutorials and end-to-end projects like:
Loading data, visualization, build models, tuning, and much more...

Finally Bring Machine Learning To Your Own Projects

Skip the Academics. Just Results.

See What's Inside

16 Responses to Non-Linear Regression in R

  1. Finn Gilling May 12, 2016 at 7:13 pm #

    Hi Jason
    Just joined your seamingly fantastic course in R and machine learning.
    I want to practise it with a colleague and therefore I will ask you if it is possible to take the 14 courses in a day or two by saving your course emails – or should we take one email course and finish it before we receive the next one?
    Regards
    Finn Gilling

    finn@gilling.com

  2. Akash October 4, 2016 at 3:37 am #

    I have data of electricity consumption for 2 days.I want to train an SVR model using this data and predict for next 1 day only but the R software predicts for 2 days instead of one day. Basically I want to train the model using more data but predict for lesser values.

    • Jason Brownlee October 4, 2016 at 7:26 am #

      Hi Akash, I think this may be how you are framing your problem rather than SVR.

      Perhaps reconsider how you have your data structure for the problem?

  3. Ramy November 2, 2018 at 3:50 am #

    Hello,
    I have a question about MARS, If I have for example 50 observations of 5 sensors with 5 signals and I tried to do regression with MARS. I found the model eliminate the 5th sensor readings as it is so near. So, the model is function of 4 sensor variables and does not be affected by the 5th one and I use this model for prediction. But if suddenly and for any reason happen that I get an observation that has a reading of the 5th sensor which is too high than that I had before so the model will not sense that however this is an indication for a fault. So, now I wonder what shall I do to keep the model at least sense that there is a problem or something like that.

    Thanks in advance,
    Ramy

    • Jason Brownlee November 2, 2018 at 5:56 am #

      Perhaps try scaling (standardizing or normalizing) the data prior to fitting the model?
      Perhaps try a suite of methods in addition to MARS?

      • Ramy November 2, 2018 at 3:41 pm #

        Thanks for your response but still there some variable that are not included in the model. In addition, I think that MARS deals with data within the training data as if a new observation which beyond the region the response is the same and nothing change.

  4. ghizlane May 16, 2019 at 6:50 am #

    thank you
    but why you don’t use the training and testing or validation part for neural network, it’s not necessary to build a neural network ?

    • Jason Brownlee May 16, 2019 at 2:21 pm #

      Sorry, I don’t understand. Can you elaborate please?

  5. ghizlane May 18, 2019 at 6:59 am #

    Hi,
    I wonder why you did not divide your database in two (training data for example(70%) and testing data(30%)), to validate the model of regression especially for neural networks

  6. ghizlane May 20, 2019 at 2:09 am #

    thank you for replying

  7. obanah June 12, 2019 at 9:18 pm #

    Thank you Jason ,
    one question about neural network

    linear output =TRUE ? is it for regression ?
    linear output = FALSE , is it for classification ?

    knowing that I am working on predictive models, using regression by neural network

    • Jason Brownlee June 13, 2019 at 6:15 am #

      I recommend checking the documentation for the function.

  8. tashkili June 13, 2019 at 4:49 am #

    Good job

    I want to ask a question about the neuralalnet package, I just find the training and testing , there is no validation in the function, how to validate the model or it is sufficient to use only training and testing in the neuralnet package

    in my case training(92%) testing(83%)

    Thank you

Leave a Reply