Last Updated on August 22, 2019
The R platform for statistical computing is perhaps the most popular and powerful platform for applied machine learning.
The caret package in R has been called “R’s competitive advantage“. It makes the process of training, tuning and evaluating machine learning models in R consistent, easy and even fun.
In this post you will discover the caret package in R, it’s key features and where to go to learn more about it.
Kick-start your project with my new book Machine Learning Mastery With R, including step-by-step tutorials and the R source code files for all examples.
Let’s get started.
What is the Caret R Package
Caret was built on a key philosophy in machine learning, that of the no free lunch theorem. The theorem states, that given no prior knowledge of prediction problem, no single method can be said to be better than any other.
In this face of this theorem, the caret package has an opinionated stance on how applied machine learning should be conducted. You cannot know which algorithm or which algorithm parameters will be optimal for a given problem, it can only be known by empirical experimentation. This is the process that the caret package was designed to facilitate.
It does this in a few key ways:
- Streamlined Model Creation: It provides a consistent interface to train a large number of the most popular third party algorithms in R.
- Evaluate the Effect of Parameters on Performance: It provides tools to grid search combinations of algorithm parameters against an objective measure to understand the effect of parameters on the model for a given problem.
- Choose an Optimal Model: It provides tools to evaluate and compare models on a given problem to locate the most suitable using objective criteria.
- Estimate Model Performance: It provides tools to estimate the accuracy of models on unseen data for a given problem.
Need more Help with R for Machine Learning?
Take my free 14-day email course and discover how to use R on your project (with sample code).
Click to sign-up and also get a free PDF Ebook version of the course.
The caret package has many features built around the core philosophy. Some examples include:
- Data Splitting: Split data in training and test datasets.
- Data Pre-processing: Prepare data for modeling such as normalization and standardization.
- Feature Selection: Methods to select only those attributes required to make effective predictions.
- Feature Importance: Evaluate the relevance of each attribute in the dataset on the predicted attribute.
- Model Tuning: Evaluate the effect of algorithm parameters on performance and locate an optimal configuration
- Parallel Processing: Tune and estimate model performance using parallel computing such as multiple cores on a workstation to give performance improvements.
- Visualization: Better understand training data, model comparison and the effect of parameters on model with tailored visualizations.
Where Did Caret Come From
Caret is a package in R created and maintained by Max Kuhn form Pfizer. Development started in 2005 and was later made open source and uploaded to CRAN.
Caret is actually an acronym which stands for Classification And REgression Training (CARET).
It was initially developed out of the need to run multiple different algorithms for a given problem. R packages are created by third parties and can vary in terms of their parameters and syntax when training and generating predictions. The first versions of the caret package were designed to unify model training and prediction.
It later expanded to further standardize related common tasks such as parameter tuning and determining variable importance.
Interview with Max Kuhn
Max Kuhn is interviewed by DataScience.LA at the useR conference. In the interview, Max talks about the development of caret and his use of R. He talks about the importance of testing multiple models on a given problem and the pain in working with multiple different packages at the same time, the impetus for creating the package.
Demonstration of Caret by Max Kuhn
Max Kuhn demonstrates caret and talks about its development and features of caret in this presentation. He touches again on the the no free lunch theorem and the need to test multiple models. The heart of the presentation is an example of a model on some churn data. He touches on estimating model performance, algorithm tuning and much more.
If you are interested in more information in the caret package for, check out some of the links below.
- Caret Package Homepage
- Caret Package on CRAN
- Caret Package Manual (PDF, all the functions)
- A Short Introduction to the caret Package (PDF vignette)
- Building Predictive Models in R Using the caret Package (PDF paper)
- Open source project on GitHub (source code)
Absolutely love everything you post on your website.
Highly recommend Max’s predictive analytics training which is basically a walkthrough of the caret package and his book.
hey Ken, Which training you refer to ? I know Max’s book, but don’t know if there is an official training course on that. Is that the videos on YouTube ? Thanks.
Jason, Thanks for sharing your knowledge on this platform. This is useful!
He’s on DataCamp now. Look for the course called “machine learning toolbox”
Absolutely good website from me. I want to ask something. Can we produce the sample of tree in random forest?
What do you mean exactly?