How Beginners Get It Wrong In Machine Learning

The 5 Most Common Mistakes That Beginners Make
And How To Avoid Them.

I help beginners get started in machine learning.

But I see the same mistakes in both mindset and action again and again.

In this post, you will discover the 5 most common ways that I see beginners slip-up when getting started in machine learning.

I firmly believe that
anyone can get started and do really well
with applied machine learning.

Hopefully, you can identify yourself in one or more of the traps below and take some corrective action to get back on course.

Let’s get started.

How NOT To Get Started With Machine Learning

How NOT To Get Started With Machine Learning
Photo by Bart Everson, some rights reserved.

1) Don’t Start With Theory

The traditional approach to teach machine learning is bottom up.

  1. Work hard to learn the background in math.
  2. Work hard to learn the theory of machine learning.
  3. Work hard to implement algorithms from scratch
  4. ??? (insert magic here)
  5. Finally start using machine learning (your goal!).

This approach is slow. It’s hard. It’s designed for academics that want to extend the state of the art.

It is not designed for practitioners that want a result.

The Trap

You know you are caught in this trap if you think or say things like:

  • I need to complete this course in linear algebra first.
  • I need to go back and get a Ph.D. first.
  • I have to read this textbook first.

The Way Out

How does learning 4 years of math or esoteric algorithm theory get you to where you want to be?

You are more likely to stop. To fail. To not get any closer to your goal.

The solution is to flip the model.

If the valuable contribution of machine learning to the market is the set of accurate predictions, then learn how to model problems and make accurate predictions. Start here.

Then get really damn good at it.

Read, steal, harness the theory if you need it, but only in service of your goal. Only if it makes you better at delivering value.

2) Don’t Study All of Machine Learning

Machine learning is a very large field of study.

It is the automation of learning processes with computers and has deep overlap with Artificial Intelligence.

From esoteric learning theory to robotics. The field is massive.

The field is way too big for you to take on all of it.

The Trap

You know you’ve succumbed to this trap if you think things like:

  • I need to learn about each new technique mentioned on a new site.
  • I need to learn about computer vision, natural language processing, speech, etc. first.
  • I need to know everything about everything.

The Way Out

Pick one small corner and focus on it.

Then narrow it down again.

The most valuable area of machine learning is predictive modeling. Creating models from data to make predictions.

Start there.

Next, focus on a type of predictive modeling that is most relevant or interesting to you.

Then stick with it.

Maybe you choose by technique, such as deep learning. Or maybe you choose by problem type, like recommender systems.

Maybe you’re not sure, so just pick one anyway. Get good or at the very least proficient.

Then, later, circle back to another area.

3) Don’t Fiddle Around With Algorithms

Machine learning is really about the algorithms.

There are a lot of algorithms. And each algorithm is a complex system and it’s own little field of study. It’s own ecosystem.

You can lose yourself in an algorithm. And people do.

They’re called academics.

The Trap

You’re in this trap if you find yourself saying:

  • I need to know why it works before I use it.
  • I need to deeply understand the hyperparameters first.
  • I need to explain the cause and effect when tuning.

The Way Out

Algorithms are not results. They are a means to a result.

In fact, machine learning algorithms are a commodity.

Swap them out. Try out a ton of them on your problem. Tune them some, but move on.

You can learn more about algorithms to get a better result, but know when to stop.

Use a systematic process. Design tuning experiments and automate their execution and analysis.

Machine learning is all about good use of algorithms but applied machine learning is not just fiddling with algorithms.

Focus on the goal of delivering a result from each project, that is a set of predictions or a model that can make them.

Wring Way Go Back

Wring Way Go Back
Photo by Dallas, some rights reserved.

4) Don’t Implement Everything From Scratch

You can learn a lot from implementing algorithms from scratch.

Sometimes you even need to implement a technique, because there is no suitable or available implementation.

But, generally, you don’t have to and you shouldn’t.

Your implementation will probably suck. Sorry.

  • It will have bugs.
  • It will be slow.
  • It will be a memory hog.
  • It will not deal with edge cases well.
  • It might even be wrong.

The Trap

You’re in this trap if:

  • You’re writing code to load a CSV file (what the hell!?)
  • You’re writing code for a standard algorithm like linear regression.
  • You’re writing code for cross-validation or hyperparameter tuning.

The Way Out

Just Stop.

  • Use a general-purpose library used by tens or hundreds of thousands of other developers that handles all the edge cases and is known to be correct.
  • Use a highly optimized library that squeezes every last cycle and every last byte of memory from your hardware.
  • Use a graphical user interface for your own projects and avoid code altogether.

Implementing everything every time you want to use it is a very slow way to get started in machine learning.

If you’re implementing for learning, then be honest with yourself and separate that from learning how to deliver value with applied machine learning.

5) Don’t Change Tools All The Time

There are a lot of great machine learning tools.

In fact, great tools, along with data availability and fast hardware is why we are seeing a Renaissance in machine learning.

But you can fall into the trap of jumping to each new tool you stumble across.

The Trap

You’re in this trap if you find yourself:

  • Using each new tool you hear about.
  • Find yourself learning a new tool or language every week or month.
  • Get half-way through learning a library and leaving it behind for a new one.

The Way Out

Learn and use new tools.

But be strategic.

Integrate new tools into your systematic process for working through machine learning problems.

You’ll be a lot more efficient in working through problems if you pick one of the large major platforms and stick with it, at least until you are good or proficient with it.

The top 3 platforms I recommend are:

There are others and there are more specialty tools if that is your area.

Follow-through is the difference between a hobbyist and a professional.

Summary

In this post, you discover the 5 most common mistakes that I see made by beginners in machine learning.

Again, they were:

  1. Don’t Start With Theory.
  2. Don’t Study All of Machine Learning.
  3. Don’t Fiddle Around With Algorithms.
  4. Don’t Implement Everything From Scratch.
  5. Don’t Change Tools All The Time.

Have you fallen into any of these traps?
Do you need help getting out?

Leave a comment, I’m here to help.

41 Responses to How Beginners Get It Wrong In Machine Learning

  1. Leo October 3, 2016 at 1:11 pm #

    Nice post Jason. Do you think “The Elements of Statistical Learning” is a good book for someone interested in applying machine learning methods to different problems?

    (I saw a similar question on quora).

    • Jason Brownlee October 3, 2016 at 3:56 pm #

      Hi Leo,

      I think “The Elements of Statistical Learning” is an excellent book. It is ideal for learning theory behind techniques.

      I don’t think it is a good book for beginners and I don’t think it is a good book for applying machine learning.

      I teach a top-down and results-first approach to machine learning (rather than starting with theory) that you can learn more about here:
      http://machinelearningmastery.com/start-here/#getstarted

  2. mosab October 3, 2016 at 3:08 pm #

    thank you, its perfect advice, honestly i found machine learning could help prediction results based on simulation and experimental result i used to have ,for that i just search from where i should start to learn. this post would help a lot and any advice for me would be highly appreciated
    thanks

  3. Lindani October 3, 2016 at 8:57 pm #

    Thanks a lot Jason. I have been doing it the wrong way all along!

  4. Antole October 4, 2016 at 12:00 am #

    I just love how you always provide clarity without which a beginner can get really frustrated pretty fast.

  5. abhishek October 4, 2016 at 2:28 am #

    Thanks for the post, Jason. Much appreciated. I am facing the exact problem. I have completed 3 Machine Learning courses offered by University of Washington on Cousera. Namely, Linear Regression, Classification, Clustering and Retrieval, covering some projects and algorithms. But now I am stuck, how do I proceed?, as I feel I need to work on projects (which can be verified by experts in the field). Can you guide me, please?

  6. Tom Albert October 4, 2016 at 12:07 pm #

    Thank for your post, It is really useful, I used to get these problems when I started to research about machine learning, but it is good now. I wish I could read this post sooner 🙂

  7. Peter October 6, 2016 at 11:36 am #

    Regarding “Use a graphical user interface for your own projects and avoid code altogether.”
    – will we see anything from you about Rapid Miner? There is a free version of that program and it seems ideal if you want to avoid code.

    • Luisotsm October 7, 2016 at 3:34 am #

      Rapid Miner is a great way of using Machine Learning throught a GUI @Peter

    • Jason Brownlee October 7, 2016 at 4:49 am #

      I may cover RapidMiner in the future Peter.

      For now ,I focus on teaching Weka as it is very easy to use and quick to get good results for beginners.

  8. yw October 6, 2016 at 2:58 pm #

    the post is valueble for me!I find I have fallen into all traps above!thank you very much!

  9. Shravan Kumar Parunandula October 6, 2016 at 5:18 pm #

    Very informative post Jason. Looking forward for the next tip. Thank you

  10. Mansoor October 6, 2016 at 5:20 pm #

    Excellant article Jason. I guess I have suffered from all five traps at times, but latest is platform change.
    Ive been using matlab for neural networks and svm with the aim to deploy my matlab code as java code for use in a web app, matlab offers this kind of deployment. Have fair bit of experience in matlab n neural network toolbox. But recently learned that neural net training functions r not supported by compiler for code deployment outside matlab. Thats a big setback. Now looking for other suitable platforms for development work with neural nets. Any suggestions? as there r plethora of options available. Application area is nlp n image recognition. Thanks.

  11. Angelo October 6, 2016 at 5:21 pm #

    Thanks Jason for the post, I was doing all the five errors and I was starting to get frustrated. I found the article really helpful.

    • Jason Brownlee October 7, 2016 at 4:46 am #

      I’m keen to hear how your progress changes Angelo.

  12. Nader October 7, 2016 at 1:55 am #

    Loving it !!!

  13. omid October 7, 2016 at 7:17 am #

    Thanks Jason for this good article

  14. Srivathsan Madhavan October 7, 2016 at 4:33 pm #

    Hi Jason,

    Very nice article – infact your blog is tremendously useful for a dummy beginner like myself.

    Sorry if I am being pedantic but I guess the “mistake” must be “Start with the Theory” and your solution / better alternative would be “Don’t Start with the Theory”. Likewise for the other 4 mistakes. Please let me know if I am mistaken.

    Cheers!
    Vatsan

    • Jason Brownlee October 8, 2016 at 10:33 am #

      Thanks Srivathsan.

      Nice – yes that would be better English, but perhaps not as easy to read/skim in a blog post.

  15. Shalini October 8, 2016 at 12:17 am #

    Hi Jason,

    I have fallen on all 5 traps. This was a timely article for me. Thank you for the post.

    • Jason Brownlee October 8, 2016 at 10:40 am #

      I’m glad to hear that it was useful Shalini.

      Hang in there!

  16. Merid October 8, 2016 at 1:03 am #

    I really appreciate your valuable advice and posts, still I afraid to be researcher on ML; is that possible to proceed without understanding theoretical background of ML? as per your summary.

    • Jason Brownlee October 8, 2016 at 10:43 am #

      Yes Merid. The understanding will come with time.

      Research is different, but if in your research machine learning is just a tool or means to an end – then treat it as such.

      The same way you treat programming languages as a tool without knowing the theory of computability.

  17. Adi October 10, 2016 at 10:29 pm #

    Great article Jason!
    However, I’ve (also) been doing it all wrong so far. And I managed to complete ~10 ML courses (coursera, edX, udacity,… you name it) and many books.
    But I got a real dataset(industrial logs) now, and it doesn’t match anything. Nothing seem to be working on it.
    What should you try first? Given that you have the tools, knowledge and will. What to apply to which data? Which are the very first steps AFTER the ETL process is done?

    Thanks
    Adi

  18. Matthew October 13, 2016 at 2:48 am #

    A great one, I love it. Thank you for sharing.

  19. Charles Green May 3, 2017 at 2:15 am #

    Hello Jason,

    Your material is great. Please keep it up!
    I understand and respect the results first approach. It makes a lot of sense.

    I also understand that a Ph.D. is not needed in ML but I still assume it helps for career progression. Outside of academia do you feel it still has value?

    I’ve recently started a new position working within Machine Learning where I can gain hands-on experience over the short-term. This works nicely with my 15 years experience as a software developer but in the long-term, I’m still wondering if a doctorates program would also be helpful over the long term but not sure if the time and effort would be better spent in other areas.

    I realize everyone’s experiences and situations are different however any thoughts would be greatly appreciated.

    Many thanks,
    Charles

    • Jason Brownlee May 3, 2017 at 7:42 am #

      Generally no.

      You can get an interview easier with a PhD, but in a good company, you get hired by what you can do, not based on a credential.

      I’ve worked alongside practitioners without higher degrees that were excellent and alongside others that had higher degrees and were terrible.

  20. Atul July 14, 2017 at 10:13 am #

    I disagree with “3) Don’t Fiddle Around With Algorithms”
    If you don’t want to diddle with algorithm then why use machine learning at all (anyway you are not able to do anything useful if you don’t know algorithm. And forget about getting hired, no matter how good your resume look like).

    • Jason Brownlee July 15, 2017 at 9:35 am #

      Fair enough.

      I see developers getting lost in one algorithm for 6 months, where in just a few weeks they could learn the whole predictive modeling lifecycle and start delivering value in business.

Leave a Reply