5 Mistakes Programmers Make when Starting in Machine Learning

There is no right way to get into machine learning. We all learn slightly different ways and have different objectives of what we want to do with or for machine learning.

A common goal is to get productive with machine learning quickly. If that is your goal then this post highlights five common mistakes programmers make on the path to quickly being productive machine learning practitioners.

Mistakes Programmers Make when Starting in Machine Learning
Photo credited to aarontait, some rights reserved.

1. Put Machine Learning on a pedestal

Machine learning is just another bag of techniques that you can use to create solutions to complex problems.

Because it is a burgeoning field, machine learning is typically communicated in academic publications and textbooks for postgraduate students. This gives it the appearance that it is elite and impenetrable.

A mindset shift is required to be effective at machine learning, from technology to process, from precision to “good enough”, but the same could be said for other complex methods that programmers are interested in adopting.

2. Write Machine Learning Code

Starting in machine learning by writing code can make things difficult because it means that you are solving at least two problems rather than one: how a technique works so that you can implement it and how to apply the technique to a given problem.

It is much easier to work on one problem at a time and leverage machine learning and statistical environments and libraries of algorithms to learn how to apply a technique to a problem. This allows you to spot check and tune a variety of algorithms relatively quickly and tune the one or two that look promising rather than investing large amounts of time interpreting ambiguous research papers containing algorithm descriptions.

Implementing an algorithm can be treated as a separate project to be completed at a later time, such as for a learning exercise or if the prototype system needs to me put into operations. Learn one thing at a time, I recommend starting with a GUI based machine learning framework whether you’re a programmer or not.

3. Doing Things Manually

A process surrounds applied machine learning including problem definition, data preparation and presentation of results, among other tasks. These processes along with the testing and tuning of algorithms can and should be automated.

Automation is a big part of modern software development for builds, tests and deployment. There is great advantage in scripting data preparation, algorithm testing and tuning and the preparation of results in order to gain the benefits of rigor and speed of improvement. Remember and reuse the lessons learned in professional software development.

The failure to start with automation (such as Makefiles or similar build system) is likely due to the fact that many programmers come to machine learning from books and courses that have less focus on the applied nature of the field. In fact, brining automation to applied machine learning is a huge opportunity for programmers.

4. Reinvent Solutions to Common Problems

Hundreds and thousands of people have likely implemented the algorithm you are implementing before you or have solved a problem type similar to the problem you are solving, exploit their lessons learned.

There is a wealth of knowledge out there of solving applied machine learning. Granted much of it may be tied up in books and research publications, but you can access it. Do your homework and search Google, Google Books, Google Scholar and reach out to the machine learning community.

If you are implementing an algorithm:

• Do you have to implement it? Can you reuse an existing open source algorithm implementation in a library or tool?
• Do you have to implement from scratch? Can you code review, learn from or port an existing open source implementation?
• Do you have to interpret the canonical algorithm description? Are there algorithm descriptions in other books, papers, theses, or blog posts that you can review and learn from?

Photo credited to justgrimes, some rights reserved

If you are addressing a problem:

• Do you have to test all algorithms on the problem? Can you exploit studies on this or similar problem instances of the same general type that suggest algorithms and algorithm classes that perform well?
• Do you have to collect your own data? Are their publicly available data sets or APIs that you can use directly or as a proxy for your problem to quickly learn which methods are likely to perform well?
• Do you have to optimize the parameters of the algorithm? Are the heuristics you can use for configuring the algorithm presented in papers or studies of the algorithm?

What would be your strategy if you have a problem with a programming library or a specific type of data structure? Use the same tactics in the field of machine learning. Reach out to the community and ask for resources that you may be able to exploit to accelerate your learning and progress on your project. Consider forums and Q&A sites to start with and contact academics and specialists as the next step.

5. Ignoring the Math

You do not need the mathematical theory to get started, but maths is a big part of machine learning. The reason for this is it provides perhaps the most efficient and unambiguous way to describe problems and the behaviors of systems.

Ignoring the mathematical treatments of algorithms can lead to problems such as having a limited understanding of a method or adopting a limited interpretation of an algorithm. For example, many machine learning algorithms have an optimization at their core that is incrementally updated. Knowing about the nature of the optimization being solved (is the function convex) allows you to use efficient optimization algorithms that exploit this knowledge.

Internalizing the mathematical treatment of algorithms is slow and comes with mastery. Particularly if you are implementing advanced algorithms from scratch including the internal optimization algorithms, take the time to learn the algorithm from mathematical perspective.

Summary

In this post you learned about 5 common mistakes that programmers make when getting started in machine learning. The five lessons are:

• Don’t put machine learning on a pedestal
• Don’t write machine learning code
• Don’t do things manually
• Don’t reinvent solutions to common problems
• Don’t ignore the math

UPDATE: Continue the conversation on HackerNews and DataTau.