How To Load CSV Machine Learning Data in Weka

You must be able to load your data before you can start modeling it.

In this post you will discover how you can load your CSV dataset in Weka. After reading this post, you will know:

  • About the ARFF file format and how it is the default way to represent data in Weka.
  • How to load a CSV file in the Weka Explorer and save it in ARFF format.
  • How to load a CSV file in the ArffViewer tool and save it in ARFF format.

This tutorial assumes that you already have Weka installed.

Let’s get started.

How To Load CSV Machine Learning Data in Weka

How To Load CSV Machine Learning Data in Weka
Photo by Thales, some rights reserved.

How to Talk About Data in Weka

Machine learning algorithms are primarily designed to work with arrays of numbers.

This is called tabular or structured data because it is how data looks in a spreadsheet, comprised of rows and columns.

Weka has a specific computer science centric vocabulary when describing data:

  • Instance: A row of data is called an instance, as in an instance or observation from the problem domain.
  • Attribute: A column of data is called a feature or attribute, as in feature of the observation.

Each attribute can have a different type, for example:

  • Real for numeric values like 1.2.
  • Integer for numeric values without a fractional part like 5.
  • Nominal for categorical data like “dog” and “cat”.
  • String for lists of words, like this sentence.

On classification problems, the output variable must be nominal. For regression problems, the output variable must be real.

Need more help with Weka for Machine Learning?

Take my free 14-day email course and discover how to use the platform step-by-step.

Click to sign-up and also get a free PDF Ebook version of the course.

Start Your FREE Mini-Course Now!

Data in Weka

Weka prefers to load data in the ARFF format.

ARFF is an acronym that stands for Attribute-Relation File Format. It is an extension of the CSV file format where a header is used that provides metadata about the data types in the columns.

For example, the first few lines of the classic iris flowers dataset in CSV format looks as follows:

The same file in ARFF format looks as follows:

You can see that directives start with the at symbol (@) and that there is one for the name of the dataset (e.g. @RELATION iris), there is a directive to define the name and datatype of each attribute (e.g. @ATTRIBUTE sepallength REAL) and there is a directive to indicate the start of the raw data (e.g. @DATA).

Lines in an ARFF file that start with a percentage symbol (%) indicate a comment.

Values in the raw data section that have a question mark symbol (?) indicate an unknown or missing value. The format supports numeric and categorical values as in the iris example above, but also supports dates and string values.

Depending on your installation of Weka, you may or may not have some default datasets in your Weka installation directory under the data/ subdirectory. These default datasets distributed with Weka are in the ARFF format and have the .arff file extension.

Load CSV Files in the ARFF-Viewer

Your data is not likely to be in ARFF format.

In fact, it is much more likely to be in Comma Separated Value (CSV) format. This is a simple format where data is laid out in a table of rows and columns and a comma is used to separate the values on a row. Quotes may also be used to surround values, especially if the data contains strings of text with spaces.

The CSV format is easily exported from Microsoft Excel, so once you can get your data into Excel, you can easily convert it to CSV format.

Weka provides a handy tool to load CSV files and save them in ARFF. You only need to do this once with your dataset.

Using the steps below you can convert your dataset from CSV format to ARFF format and use it with the Weka workbench. If you do not have a CSV file handy, you can use the iris flowers dataset. Download the file from the UCI Machine Learning repository (direct link) and save it to your current working directory as iris.csv.

1. Start the Weka chooser.

Weka GUI Chooser

Screenshot of the Weka GUI Chooser

2. Open the ARFF-Viewer by clicking “Tools” in the menu and select “ArffViewer”.

3. You will be presented with an empty ARFF-Viewer window.

Weka ARFF Viewer

Weka ARFF Viewer

4. Open your CSV file in the ARFF-Viewer by clicking the “File” menu and select “Open”. Navigate to your current working directory. Change the “Files of Type:” filter to “CSV data files (*.csv)”. Select your file and click the “Open” button.

Load CSV In ARFF Viewer

Load CSV In ARFF Viewer

5. You should see a sample of your CSV file loaded into the ARFF-Viewer.

6. Save your dataset in ARFF format by clicking the “File” menu and selecting “Save as…”. Enter a filename with a .arff extension and click the “Save” button.

You can now load your saved .arff file directly into Weka.

Note, the ARFF-Viewer provides options for modifying your dataset before saving. For example you can change values, change the name of attributes and change their data types.

It is highly recommended that you specify the names of each attribute as this will help with analysis of your data later. Also, make sure that the data types of each attribute are correct.

Load CSV Files in the Weka Explorer

You can also load your CSV files directly in the Weka Explorer interface.

This is handy if you are in a hurry and want to quickly test out an idea.

This section shows you how you can load your CSV file in the Weka Explorer interface. You can use the iris dataset again, to practice if you do not have a CSV dataset to load.

1. Start the Weka GUI Chooser.

2. Launch the Weka Explorer by clicking the “Explorer” button.

Weka Explorer

Screenshot of the Weka Explorer

3. Click the “Open file…” button.

4. Navigate to your current working directory. Change the “Files of Type” to “CSV data files (*.csv)”. Select your file and click the “Open” button.

You can work with the data directly. You can also save your dataset in ARFF format by clicking he “Save” button and typing a filename.

Use Excel for Other File Formats

If you have data in another format, load it in Microsoft Excel first.

It is common to get data in another format such as CSV using a different delimiter or fixed width fields. Excel has powerful tools for loading tabular data in a variety of formats. Use these tools and first load your data into Excel.

Once you have loaded your data into Excel, you can export it into CSV format. You can then work with it in Weka, either directly or by first converting it to ARFF format.

Resources

Below are some additional resources that you may find useful when working with CSV data in Weka.

Summary

In this post you discovered how to load your CSV data into Weka for machine learning.

Specifically, you learned:

  • About the ARFF file format and how Weka uses it to represent datasets for machine learning.
  • How to load your CSV data using ARFF-Viewer and save it into ARFF format.
  • How to load your CSV data directly in the Weka Explorer and use it for modeling.

Do you have any questions about loading data in Weka or about this post? Ask your questions in the comments and I will do my best to answer.


Want Machine Learning Without The Code?

Master Machine Learning With Weka

Develop Your Own Models in Minutes

…with just a few a few clicks

Discover how in my new Ebook:
Machine Learning Mastery With Weka

Covers self-study tutorials and end-to-end projects like:
Loading data, visualization, build models, tuning, and much more…

Finally Bring The Machine Learning To
Your Own Projects

Skip the Academics. Just Results.

Click to learn more.


28 Responses to How To Load CSV Machine Learning Data in Weka

  1. alcajerah June 23, 2016 at 10:13 pm #

    great post

  2. Ayo October 25, 2016 at 11:26 am #

    Thank you for this helpful post. please I encountered java.io.IOException when trying to view my .CSV file in WEKA. I need your help.

    • Jason Brownlee October 26, 2016 at 8:29 am #

      I’m sorry to hear that Ayo.

      Maybe your file is too large?

      Maybe try loading it in excel first and check that the file is not corrupt?

  3. Sunkanmi November 29, 2016 at 12:31 pm #

    Hi Jason,
    Thanks for the good job you’re doing.

    What is the limit of the size of csv file that can be converted into arff format by the weka workbench.
    I have a csv file above 2gig and I couldn’t get weka Arff viewer or explorer to load it up for conversion.

  4. Rodney Williams December 17, 2016 at 11:20 am #

    I get an error message, “java.io.IOE.exception 12 Problem encountered on line:2”

    same messagewhether I try to load with tools or directly using Explorer

    • Jason Brownlee December 18, 2016 at 5:28 am #

      Sorry to hear that Rodney. I’ve not seen this before. Perhaps try posting to stack overflow or the Weka email list?

  5. Farah Sardouk December 23, 2016 at 4:04 am #

    Thank you!

  6. santanu chatterjee January 28, 2017 at 7:52 pm #

    I have a data set which contains quantity,unit price,duty,model,sepcs,brand names of some products. I initially used Naive Bayes and later used auto weka to determine that my ideal algorithm will be random forest and its gives 10% accuracy rate and runs smoothly,whenever I use test using training data set.

    All hell breaks loose, when I copy the same data file and remove the brand names,so that I can use it as a test dataset. I either receive inputmappedclassifier option or errors like – “problem evaluating classifier-index4-size1 or “different nos of lables 15!=1.

    Namely, I have placed brand name column for both test and training datasets at the end.I did some online search and used question mark on the brand column of the test data set. I have used random alphanumeric values. I have even converted them into arff format and cross checked the attribute data type on both file,but nothing seems to help. Where am I going wrong?I know I am close.can anyone help?

  7. Asmaa M. Elmohamady January 29, 2017 at 7:55 am #

    Dear Jason,

    i have two problems when load .csv file to weka

    1- The file loaded correctly but because of enclosure character the sentences are not loaded completely.

    2- When trying to apply stringTowordvector filter this error appear (attribute names are not unique! causes: ‘class’).

    • Jason Brownlee February 1, 2017 at 10:12 am #

      Hi Asmaa, sorry, I don’t have good examples of loading textual data in Weka.

  8. vidhi February 22, 2017 at 3:12 pm #

    i have multiple values for a single attribute. how can i write them

    • Jason Brownlee February 23, 2017 at 8:45 am #

      Great question!

      Consider making them binary attributes. Have one column for each value, then mark a 1 or a 0 on a row for each column for a given observation.

      I hope that helps.

  9. Toby March 15, 2017 at 7:19 am #

    Hi Jason,

    Great post! Just curious, is it possible to set this up real time? In that if I have a process that collects data real time via a CSV file that then needs to be converted to arff. Can this be done?

    And then once this is done, is it possible to add and delete values from the arff object?

    thanks!

    • Jason Brownlee March 15, 2017 at 8:17 am #

      I would recommend using the command line interface for Weka to achieve this.

      Sorry, I don’t have examples, but consider checking the Weka system/user guide provided with Weka.

  10. Boum March 21, 2017 at 10:44 am #

    How can i introduce 3 indpendent real attributes classes ?
    ie: Not one class that contains 3 values No; 3 indpendent real attributes classes

    • Jason Brownlee March 22, 2017 at 7:54 am #

      I believe Weka does not support outputting a vector prediction (e.g. 3 classes).

  11. nicolas April 28, 2017 at 6:12 am #

    Hello jason,

    I have built and followed the steps on this page to transform my CSV file into an arff successfully. However, now that I load the dataset into Weka, it is impossible for me to use a j48 tree. What can I do to build a tree and begin analyzing my data? Thanks!

    Best,
    Nick

    • Jason Brownlee April 28, 2017 at 7:57 am #

      Perhaps you need to make your output variable nominal so you can use classification algorithms?

  12. Dimple April 28, 2017 at 9:26 pm #

    the first row values are taken as attribute names, thus losing first row values. How can we retain the values and give explicit attribute names?

    • Jason Brownlee April 29, 2017 at 7:25 am #

      Good question. I’m not sure off-hand, sorry.

      One way would be to specify col names in your file. Or even add a dummy first line to the file.

  13. reda June 9, 2017 at 11:46 pm #

    i tray to open my csv data in weka but is not work erreur [[ file’data.csv’ not recognised as an csv data files file reaon : 1 problem encountered on line 2

    • Jason Brownlee June 10, 2017 at 8:24 am #

      Consider double checking your file is a CSV file. Open in a text editor or MS Excel to confirm.

  14. marzieh July 26, 2017 at 10:59 pm #

    Deer Mr.Brownlee,

    I going to use weka for a IDS purpose on KDDcup99 dataset.
    I have to use SVM and Entropy on this dataset ,and according to this result I will decrease some attributes from the dataset and make a decision tree for detecting intrusion faster ,
    let me know is it possible to use WEKA GUI? or I should use weka library in java?

    • Jason Brownlee July 27, 2017 at 8:07 am #

      Sorry, I have not used that dataset in many years. My best advice is to try.

Leave a Reply