Last Updated on December 13, 2019
Before you can build machine learning models, you need to load your data into memory.
In this post you will discover how to load data for machine learning in Python using scikit-learn.
Kick-start your project with my new book Machine Learning Mastery With Python, including step-by-step tutorials and the Python source code files for all examples.
Let’s get started.
- Update March/2018: Added alternate link to download the dataset as the original appears to have been taken down.
Load CSV Data
Photo by Jim Makos, some rights reserved
Packaged Datasets
The scikit-learn library is packaged with datasets. These datasets are useful for getting a handle on a given machine learning algorithm or library feature before using it in your own work.
This recipe demonstrates how to load the famous Iris flowers dataset.
|
1 2 3 4 |
# Load the packaged iris flowers dataset # Iris flower dataset (4x150, reals, multi-label classification) iris = load_iris() print(iris) |
Load from CSV
It is very common for you to have a dataset as a CSV file on your local workstation or on a remote server.
This recipe show you how to load a CSV file from a URL, in this case the Pima Indians diabetes classification dataset.
You can learn more about the dataset here:
From the prepared X and y variables, you can train a machine learning model.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
# Load the Pima Indians diabetes dataset from CSV URL import numpy as np import urllib # URL for the Pima Indians Diabetes dataset (UCI Machine Learning Repository) url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv" # download the file raw_data = urllib.urlopen(url) # load the CSV file as a numpy matrix dataset = np.loadtxt(raw_data, delimiter=",") print(dataset.shape) # separate the data from the target attributes X = dataset[:,0:7] y = dataset[:,8] |
Summary
In this post you discovered that the scikit-learn method comes with packaged data sets including the iris flowers dataset. These datasets can be loaded easily and used for explore and experiment with different machine learning models.
You also saw how you can load CSV data with scikit-learn. You learned a way of opening CSV files from the web using the urllib library and how you can read that data as a NumPy matrix for use in scikit-learn.
Discover Fast Machine Learning in Python!
Develop Your Own Models in Minutes
...with just a few lines of scikit-learn code
Learn how in my new Ebook:
Machine Learning Mastery With Python
Covers self-study tutorials and end-to-end projects like:
Loading data, visualization, modeling, tuning, and much more...
Finally Bring Machine Learning To
Your Own Projects
Skip the Academics. Just Results.
It’s great to see how to load data from a URL. Do you have an example of how to actually load text from a file?
I think there is a small bug.
When code separating features to X array, it is missing the 8ths features.
So the twelfth line should be : X = dataset[:,0:8]
This way it will consist last features right before the targets given in the dataset.
Hi, thanks for this. Just a note, for Python 3*, it should be “import urllib.request” with “raw_data = urllib.request.urlopen(url)” and for Python 2* it should be “import urllib2” with “raw_data = urllib2.urlopen(url)”.
I am preparing a file for scikit learn and I would like to know how to build this file, I have my instances, features and classes. I am not sure how to make the file that is uploaded with scikit learn. Could you please clarify that?
neither do I. do you know how to build the file for scikit learn now? … i really need you help…
great blog! thanks helped a lot
Thanks.
hello sir i am a new bee to the data science i have gon through the books written by you,like machie learning mastery,machine learning algorithms from scratch and master machine learnong algorithms,i have gon through the books,next steps what i need to follow please guide me
Perhaps focus on developing a portfolio of completed projects:
https://machinelearningmastery.com/build-a-machine-learning-portfolio/
glad to have your reply sir,thank u very much
sir will you please suggest me some data science projects with python for a beginner
Yes, see here:
https://machinelearningmastery.com/practice-machine-learning-with-small-in-memory-datasets-from-the-uci-machine-learning-repository/
Thank you sir . It helped a lot.
You’re welcome.
thanks a lot sir
Dear Jason,
Great blog, thank you for that! But, it seems that Tarik is right (see his comment above). I would just have made a mistake because I applied your code. Good that I checked what the output of this operation really is. It somehow seems that when you specify the array like
X = dataset[:,0:8] the last column is actually not included in the resulting array! So it acutally goes from 0-7 (this is what you want!). If you write X = dataset[:,0:7] then you are missing the 8-th column! The next line is correct y = dataset[:,8] this is the 9th column!
Please, consider editing the code.
Best
Yuliyan
I believe the code is correct. The range 0:8 selects column 0 to 7 (stops before 8). Confirms this by printing the result.
You can learn more about slicing and ranges here:
https://machinelearningmastery.com/index-slice-reshape-numpy-arrays-machine-learning-python/
any ideas on how to learn these tutorials of iris dataset without using url , ideally i would like to exchange to a custom dataset locally to understand the code better , any help is appreciated.
Yes, you can download the dataset to your workstation and load the local file in the same manner.
Also, this may help:
https://machinelearningmastery.com/load-machine-learning-data-python/