Take Control By Creating Targeted Lists of Machine Learning Algorithms

Any book on machine learning will list and describe dozens of machine learning algorithms.

Once you start using tools and libraries you will discover dozens more. This can really wear you down, if you think you need to know about every possible algorithm out there.

A simple trick to tackle this feeling and take some control back is to make lists of machine learning algorithms.

This ridiculously simple tactic can give you a lot of power. You can use it to give you a list of methods to try when tackling a whole new class of problem. It can also give you a list of ideas when you get stuck on a dataset or your favorite method does not give you good results.

In this post you will discover the benefits of creating lists of machine learning algorithms, how to do it, how to do it well and why you should start creating your first list of algorithms today.

Kick-start your project with my new book Master Machine Learning Algorithms, including step-by-step tutorials and the Excel Spreadsheet files for all examples.

Create a List of Machine Learning Algorithms
Photo by Joel Montes de Oca, some rights reserved

Dealing with So Many Algorithms

There are hundreds of machine learning algorithms.

I see this leading to two problems:

1. Overwhelm

The fact that there are so many algorithms to choose from and try on a given machine learning problem causes some people to freeze up and do nothing.

The fact is, you don’t need to get the best result, you only need a result – a beachhead on the problem – and you can get there by spot checking a few algorithms.

2. Favorites

Because there are so many algorithms, some people select one or two favorite algorithms and only use them. This limits the results they can achieve and the problems that they can address.

Favorites are dangerous. Some algorithms are more powerful than others, but that power comes at a cost of complexity and parsimony. They are tools, leave your emotional attachment at the door.

Get your FREE Algorithms Mind Map

Sample of the handy machine learning algorithms mind map.

I've created a handy mind map of 60+ algorithms organized by type.

Download it, print it and use it. 

Take Control of the Algorithms

You need focus, a starting point to address the problem of dealing with so many machine learning algorithms.

This involves finding the edges and pushing back the fog on what is out there and what you can use when. This will give you a sense of control over the algorithms and help you to wield them rather than make you feel overwhelmed.

The great thing is, you don’t need to become an expert in each algorithm to make progress. You don’t need to know very much about each algorithm at all.

Collecting simple information such as algorithm names and the general problems to which they are applicable can help you quickly and confidently build familiarization and confidence with the extent of machine learning algorithms available.

How to Build and Maintain a List of Algorithms

The answer is to build your own personal list of machine learning algorithms.

I’m a list maker, and this method really lights up my brain.

Open a text file, word document or spreadsheet and start listing down the names of algorithms. It’s that simple. You can also list the general class to which the algorithm belongs and the general types of problems that it can address.

Define your own categories. This list is a tool to help you understand and navigate the machine learning tools at your disposal. Customize the list to include the algorithm details that you care about.

Examples of Algorithm Lists To Create

Below are 10 examples of machine learning algorithm lists that you could create.

• Regression algorithms
• SVM algorithms
• Data projection algorithms
• Deep learning algorithms
• Time series forecasting algorithms
• Rating system algorithms
• Recommender system algorithms
• Feature selection algorithms
• Class imbalance algorithms
• Decision tree algorithms

Tips for Great Algorithm Lists

Creating a list of algorithms is relatively easy. The hard part is knowing why you want the list. The “why” will help you define the type of list you want to create and the algorithm properties you want to describe in your list.

Start with the current project you are working on or your current interests. For example, if you are working on a time series or image classification problem, list all the algorithms that you could apply to that problem. If you are deeply interested in Support Vector Machines, list all the variations of SVM that you can find.

Don’t try to create the perfect list in one sitting. Create it and keep adding to it over days and weeks. It is a useful resource that you can turn back to again and again and add to as your knowledge and experience grows.

In summary, 5 tips for creating great algorithm lists are:

• Start with why you want the list and use that to define the type of list to create.
• Only capture the algorithm properties you actually need, keep it as simple as possible.
• Start with a current project or interest and create a list of related algorithms.
• Don’t aim for abstract perfection, the list is for you and your needs alone.
• Add to your list over time and expand it as your skills and experience grow.

When To Use An Algorithm List

Algorithm lists are more valuable than you think.

For example, you can use it as a technique when tackling a problem type that you have never worked on before, such as recommender systems, face detection or rating systems. A simple algorithm list gives you a list of things to try.

When working on a familiar problem, your prior biases often limit the results that you can achieve. A list of algorithms relevant to a problem domain can get you unstuck and even push you to achieve new and better results. That does not mean you should try all algorithms you can find, you still require reasoned and systematic application. Nevertheless, a list can provide a useful starting point.

Algorithm lists are a tool, but you can take them further. To make effective use of machine learning algorithms, you need to study them, research them, and even describe them. This is a natural extension for the algorithm list method and your lists can provide the basis for your self-study curriculum.

You could start by collecting additional properties about each algorithm and expand your list into a mini-encyclopaedia of algorithms, with one page per algorithm. I use an algorithm description template and focus on template elements that I will find useful as I refer back to the descriptions in the future, such as pseudo code and usage heuristics.

In summary, 3 examples of when you can use an algorithm list are:

• When you start working on a new class of problem.
• When you are stuck or looking for algorithms to try on an existing problem.
• When you are looking for algorithms to describe in more detail or research.

Anyone Can Create Machine Learning Algorithm Lists

You don’t need to deep dive into machine learning textbooks or open source libraries. A simple google search or browse of wikipedia will uncover many algorithm names to start-off your list.

If you are stuck on what to create for your first list, pick one of the examples above or browse a site like DataTau and pick a class of algorithm to list mentioned in an article or article title.

Again, you do not have to list every algorithm that you could list, narrow your scope to those algorithms in the libraries and tools you prefer. You don’t need to list every permutation of every algorithm, for example, you could focus on one aspect of an algorithm, such as the kernel functions for an SVM or the transfer functions for a neural network.

Don’t list all possible features of each algorithm. Stick to just the name and maybe the general class of algorithm and general types of problems for which it can be used. If you want to go deeper into an algorithm, consider the algorithm description method and template described previously.

You don’t need to understand the algorithms yet and you don’t need to be an academic. This is a beachhead that you are taking to expand your idea of what is out there, to overcome overwhelm and to finally to provide a point of departure on your journey deeper into applied machine learning.

Action Steps

In this post you learned about the simple tactic of creating lists of machine learning algorithms.

You discovered that this simple tactic can help you to overcome algorithm overwhelm and to help get you unstuck from the dangers of having favorite algorithms.

Your action step for this post is to create your first algorithm list. Pick something small, like a subclass of an algorithm. Pick something fun, like an algorithm that is hot right now.

Share your list if you like (or what your list is about), it would help to motivate others.

Discover How Machine Learning Algorithms Work!

See How Algorithms Work in Minutes

...with just arithmetic and simple examples

Discover how in my new Ebook:
Master Machine Learning Algorithms

It covers explanations and examples of 10 top algorithms, like:
Linear Regression, k-Nearest Neighbors, Support Vector Machines and much more...

Finally, Pull Back the Curtain on
Machine Learning Algorithms

Skip the Academics. Just Results.

See What's Inside