Last Updated on October 3, 2016
The 5 Most Common Mistakes That Beginners Make
And How To Avoid Them.
I help beginners get started in machine learning.
But I see the same mistakes in both mindset and action again and again.
In this post, you will discover the 5 most common ways that I see beginners slip-up when getting started in machine learning.
I firmly believe that
anyone can get started and do really well
with applied machine learning.
Hopefully, you can identify yourself in one or more of the traps below and take some corrective action to get back on course.
Let’s get started.
1) Don’t Start With Theory
The traditional approach to teach machine learning is bottom up.
- Work hard to learn the background in math.
- Work hard to learn the theory of machine learning.
- Work hard to implement algorithms from scratch
- ??? (insert magic here)
- Finally start using machine learning (your goal!).
This approach is slow. It’s hard. It’s designed for academics that want to extend the state of the art.
It is not designed for practitioners that want a result.
You know you are caught in this trap if you think or say things like:
- I need to complete this course in linear algebra first.
- I need to go back and get a Ph.D. first.
- I have to read this textbook first.
The Way Out
How does learning 4 years of math or esoteric algorithm theory get you to where you want to be?
You are more likely to stop. To fail. To not get any closer to your goal.
The solution is to flip the model.
If the valuable contribution of machine learning to the market is the set of accurate predictions, then learn how to model problems and make accurate predictions. Start here.
Then get really damn good at it.
Read, steal, harness the theory if you need it, but only in service of your goal. Only if it makes you better at delivering value.
2) Don’t Study All of Machine Learning
Machine learning is a very large field of study.
It is the automation of learning processes with computers and has deep overlap with Artificial Intelligence.
From esoteric learning theory to robotics. The field is massive.
The field is way too big for you to take on all of it.
You know you’ve succumbed to this trap if you think things like:
- I need to learn about each new technique mentioned on a new site.
- I need to learn about computer vision, natural language processing, speech, etc. first.
- I need to know everything about everything.
The Way Out
Pick one small corner and focus on it.
Then narrow it down again.
The most valuable area of machine learning is predictive modeling. Creating models from data to make predictions.
Next, focus on a type of predictive modeling that is most relevant or interesting to you.
Then stick with it.
Maybe you choose by technique, such as deep learning. Or maybe you choose by problem type, like recommender systems.
Maybe you’re not sure, so just pick one anyway. Get good or at the very least proficient.
Then, later, circle back to another area.
3) Don’t Fiddle Around With Algorithms
Machine learning is really about the algorithms.
There are a lot of algorithms. And each algorithm is a complex system and it’s own little field of study. It’s own ecosystem.
You can lose yourself in an algorithm. And people do.
They’re called academics.
You’re in this trap if you find yourself saying:
- I need to know why it works before I use it.
- I need to deeply understand the hyperparameters first.
- I need to explain the cause and effect when tuning.
The Way Out
Algorithms are not results. They are a means to a result.
In fact, machine learning algorithms are a commodity.
Swap them out. Try out a ton of them on your problem. Tune them some, but move on.
You can learn more about algorithms to get a better result, but know when to stop.
Use a systematic process. Design tuning experiments and automate their execution and analysis.
Machine learning is all about good use of algorithms but applied machine learning is not just fiddling with algorithms.
Focus on the goal of delivering a result from each project, that is a set of predictions or a model that can make them.
4) Don’t Implement Everything From Scratch
You can learn a lot from implementing algorithms from scratch.
Sometimes you even need to implement a technique, because there is no suitable or available implementation.
But, generally, you don’t have to and you shouldn’t.
Your implementation will probably suck. Sorry.
- It will have bugs.
- It will be slow.
- It will be a memory hog.
- It will not deal with edge cases well.
- It might even be wrong.
You’re in this trap if:
- You’re writing code to load a CSV file (what the hell!?)
- You’re writing code for a standard algorithm like linear regression.
- You’re writing code for cross-validation or hyperparameter tuning.
The Way Out
- Use a general-purpose library used by tens or hundreds of thousands of other developers that handles all the edge cases and is known to be correct.
- Use a highly optimized library that squeezes every last cycle and every last byte of memory from your hardware.
- Use a graphical user interface for your own projects and avoid code altogether.
Implementing everything every time you want to use it is a very slow way to get started in machine learning.
If you’re implementing for learning, then be honest with yourself and separate that from learning how to deliver value with applied machine learning.
5) Don’t Change Tools All The Time
There are a lot of great machine learning tools.
In fact, great tools, along with data availability and fast hardware is why we are seeing a Renaissance in machine learning.
But you can fall into the trap of jumping to each new tool you stumble across.
You’re in this trap if you find yourself:
- Using each new tool you hear about.
- Find yourself learning a new tool or language every week or month.
- Get half-way through learning a library and leaving it behind for a new one.
The Way Out
Learn and use new tools.
But be strategic.
Integrate new tools into your systematic process for working through machine learning problems.
You’ll be a lot more efficient in working through problems if you pick one of the large major platforms and stick with it, at least until you are good or proficient with it.
The top 3 platforms I recommend are:
There are others and there are more specialty tools if that is your area.
Follow-through is the difference between a hobbyist and a professional.
In this post, you discover the 5 most common mistakes that I see made by beginners in machine learning.
Again, they were:
- Don’t Start With Theory.
- Don’t Study All of Machine Learning.
- Don’t Fiddle Around With Algorithms.
- Don’t Implement Everything From Scratch.
- Don’t Change Tools All The Time.
Have you fallen into any of these traps?
Do you need help getting out?
Leave a comment, I’m here to help.
Nice post Jason. Do you think “The Elements of Statistical Learning” is a good book for someone interested in applying machine learning methods to different problems?
(I saw a similar question on quora).
I think “The Elements of Statistical Learning” is an excellent book. It is ideal for learning theory behind techniques.
I don’t think it is a good book for beginners and I don’t think it is a good book for applying machine learning.
I teach a top-down and results-first approach to machine learning (rather than starting with theory) that you can learn more about here:
thank you, its perfect advice, honestly i found machine learning could help prediction results based on simulation and experimental result i used to have ,for that i just search from where i should start to learn. this post would help a lot and any advice for me would be highly appreciated
Hi mosab, thanks.
I would recommend starting right here:
I hope that helps.
Great advice, wonderful article.
Not just machine learning but all of Science is about making predictions. Theories have only one test, predictions.
I guess so, if you frame explainability as prediction.
Thanks a lot Jason. I have been doing it the wrong way all along!
I’m glad you found it useful Lindani.
I just love how you always provide clarity without which a beginner can get really frustrated pretty fast.
Thanks Antole, I’m glad you like the post.
Thanks for the post, Jason. Much appreciated. I am facing the exact problem. I have completed 3 Machine Learning courses offered by University of Washington on Cousera. Namely, Linear Regression, Classification, Clustering and Retrieval, covering some projects and algorithms. But now I am stuck, how do I proceed?, as I feel I need to work on projects (which can be verified by experts in the field). Can you guide me, please?
Wow, well done Abhishek!
I do recommend working through problems and building up a portfolio of completed projects.
Use standard machine learning datasets and you do not need an expert to review, just compare your results.
See these notes on exactly how to get started:
See this 5-step process on exactly how to work through a practice machine learning problem:
Consider using Python if you’re a programmer:
I hope that helps as a start, shoot me an email any time if you have questions. I’m here to help.
Thank for your post, It is really useful, I used to get these problems when I started to research about machine learning, but it is good now. I wish I could read this post sooner 🙂
I’m glad to hear it was useful Tom.
Regarding “Use a graphical user interface for your own projects and avoid code altogether.”
– will we see anything from you about Rapid Miner? There is a free version of that program and it seems ideal if you want to avoid code.
Rapid Miner is a great way of using Machine Learning throught a GUI @Peter
I may cover RapidMiner in the future Peter.
For now ,I focus on teaching Weka as it is very easy to use and quick to get good results for beginners.
the post is valueble for me！I find I have fallen into all traps above！thank you very much！
I’m glad to hear you found it useful yw.
Very informative post Jason. Looking forward for the next tip. Thank you
Excellant article Jason. I guess I have suffered from all five traps at times, but latest is platform change.
Ive been using matlab for neural networks and svm with the aim to deploy my matlab code as java code for use in a web app, matlab offers this kind of deployment. Have fair bit of experience in matlab n neural network toolbox. But recently learned that neural net training functions r not supported by compiler for code deployment outside matlab. Thats a big setback. Now looking for other suitable platforms for development work with neural nets. Any suggestions? as there r plethora of options available. Application area is nlp n image recognition. Thanks.
Hi Mansoor, I think Keras is the best platform for neural networks at the moment.
You can do small stuff all the way up to very large nets, whilst all the time exploiting the efficiencies of TensorFlow and Theano backends.
See this getting started tutorial:
Thanks Jason for the post, I was doing all the five errors and I was starting to get frustrated. I found the article really helpful.
I’m keen to hear how your progress changes Angelo.
Loving it !!!
I’m glad to hear that Nader
Thanks Jason for this good article
I’m glad you found it valuable omid.
Very nice article – infact your blog is tremendously useful for a dummy beginner like myself.
Sorry if I am being pedantic but I guess the “mistake” must be “Start with the Theory” and your solution / better alternative would be “Don’t Start with the Theory”. Likewise for the other 4 mistakes. Please let me know if I am mistaken.
Nice – yes that would be better English, but perhaps not as easy to read/skim in a blog post.
I have fallen on all 5 traps. This was a timely article for me. Thank you for the post.
I’m glad to hear that it was useful Shalini.
Hang in there!
I really appreciate your valuable advice and posts, still I afraid to be researcher on ML; is that possible to proceed without understanding theoretical background of ML? as per your summary.
Yes Merid. The understanding will come with time.
Research is different, but if in your research machine learning is just a tool or means to an end – then treat it as such.
The same way you treat programming languages as a tool without knowing the theory of computability.
Great article Jason!
However, I’ve (also) been doing it all wrong so far. And I managed to complete ~10 ML courses (coursera, edX, udacity,… you name it) and many books.
But I got a real dataset(industrial logs) now, and it doesn’t match anything. Nothing seem to be working on it.
What should you try first? Given that you have the tools, knowledge and will. What to apply to which data? Which are the very first steps AFTER the ETL process is done?
Hi Adi, this process for working through predictive modeling problems may help:
A great one, I love it. Thank you for sharing.
I’m glad to hear it Matthew.
Your material is great. Please keep it up!
I understand and respect the results first approach. It makes a lot of sense.
I also understand that a Ph.D. is not needed in ML but I still assume it helps for career progression. Outside of academia do you feel it still has value?
I’ve recently started a new position working within Machine Learning where I can gain hands-on experience over the short-term. This works nicely with my 15 years experience as a software developer but in the long-term, I’m still wondering if a doctorates program would also be helpful over the long term but not sure if the time and effort would be better spent in other areas.
I realize everyone’s experiences and situations are different however any thoughts would be greatly appreciated.
You can get an interview easier with a PhD, but in a good company, you get hired by what you can do, not based on a credential.
I’ve worked alongside practitioners without higher degrees that were excellent and alongside others that had higher degrees and were terrible.
I disagree with “3) Don’t Fiddle Around With Algorithms”
If you don’t want to diddle with algorithm then why use machine learning at all (anyway you are not able to do anything useful if you don’t know algorithm. And forget about getting hired, no matter how good your resume look like).
I see developers getting lost in one algorithm for 6 months, where in just a few weeks they could learn the whole predictive modeling lifecycle and start delivering value in business.
Good advice for beginners. Thanks
Thanks Jason for the knowledge you have shared. am sure beginners like me have gained a lot. am glad.
I am a newbie to machine learning …How should I start learning it??
You can start here:
Nice work “how to start machine learning”
You can start here:
great post, I fell into all the traps 🙁
I was trying to learn everything about latest “fantastic” machine learning algo, I followed courses, read many books etc… I wanted to learn all in deep…..
but, as you said above… I’m stuck now. I’m in info overload.
Therefore, after reading your post, I stopped and breathed deeply, I want restart again.
Now, I’d like to learn about recommender systems in deep.
Can you suggest me some good intemediate/advanced resources (books, courses) about it, ?
Even though my “love” is about apply AI to financial world, but I think (if I’m not wrong) that is much difficult to apply to real world.
Any help is appreciated.
Thanks a lot.
Happy to hear the post helped.
Yes, this book is great: