Technical topics like mathematics, physics, and even computer science are taught using a bottom-up approach.
This approach involves laying out the topics in an area of study in a logical way with a natural progression in complexity and capability.
The problem is, humans are not robots executing a learning program. We require motivation, excitement, and most importantly, a connection of the topic to tangible results.
Useful skills we use every day like reading, driving, and programming were not learned this way and were in fact learned using an inverted top-down approach. This top-down approach can be used to learn technical subjects directly such as machine learning, which can make you a lot more productive a lot sooner, and be a lot of fun.
In this post, you will discover the concrete difference between the top-down and bottom-up approaches to learning technical material and why this is the approach that practitioners should use to learn machine learning and even related mathematics.
After reading this post, you will know:
- The bottom-up approach used in universities to teach technical subjects and the problems with it.
- How people learn to read, drive, and program in a top-down manner and how the top-down approach works.
- The frame of machine learning and even mathematics using the top-down approach to learning and how to start to make rapid progress as a practitioner.
Let’s get started.
You’re Doing it Wrong. Why Machine Learning Does Not Have to Be So Hard
Photo by popofatticus, some rights reserved.
Overview
This is an important blog post, because I think it can really help to shake you out of the bottom-up, university-style way of learning machine learning.
This post is divided into seven parts; they are:
- Bottom-Up Learning
- Learning to Read
- Learning to Drive
- Learning to Code
- Top-Down Learning
- Learn Machine Learning
- Learning Mathematics
Bottom-Up Learning
Take a field of study, such as mathematics.
There is a logical way to lay out the topics in mathematics that build on each other and lead through a natural progression in skills, capability, and understanding.
The problem is, this logical progression might only make sense to those who are already on the other side and can intuit the relationships between the topics.
Most of school is built around this bottom-up natural progression through material. A host of technical and scientific fields of study are taught this way.
Think back to high-school or undergraduate studies and the fundamental fields you may have worked through: examples such as:
- Mathematics, as mentioned.
- Biology.
- Chemistry.
- Physics.
- Computer Science.
Think about how the material was laid out, week-by-week, semester-by-semester, year-by-year. Bottom-up, logical progression.
The problem is, the logical progression through the material may not be the best way to learn the material in order to be productive.
We are not robots executing a learning program. We are emotional humans that need motivation, interest, attention, encouragement, and results.
You can learn technical subjects from the bottom-up, and a small percentage of people do prefer things this way, but it is not the only way.
Now, if you have completed a technical subject, think back to how to you actually learned it. I bet it was not bottom-up.
Learning to Read
Think back; how did you learn to read?
My son is starting to read. Without thinking too much, here are the general techniques he’s using (really the school and us as parents):
- Start by being read to in order to generate interest and show benefits.
- Get the alphabet down and making the right sounds.
- Memorize the most frequent words, their sounds, and how to spell them.
- Learn the “spell-out-the-word” heuristic to deal with unknown words.
- Read through books with supervision.
- Read through books without supervision.
It is important that he continually knows why reading is important, connected to very tangible things he wants to do, like:
- Read captions on TV shows.
- Read stories on topics he loves, like Star Wars.
- Read signs and menus when we are out and about.
- So on…
It is also important that he gets results that he can track and in which he can see improvement.
- Larger vocabulary.
- Smoother reading style
- Books of increasing complexity.
Here’s how he did not learn to read:
- Definitions of word types (verbs, nouns, adverbs, etc.)
- Rules of grammar.
- Rules of punctuation.
- Theory of human languages.
Learning to Drive
Do you drive?
It’s cool if you don’t, but most adults do out of necessity. Society and city design is built around personal mobility.
How did you learn to drive?
I remember some written tests and maybe a test on a computer. I have no memory of studying for them, though I very likely did. Here’s what I do remember.
I remember hiring a driving instructor and doing driving lessons. Every single lesson was practical, in the car, practicing the skill I was required to master, driving the vehicle in traffic.
Here’s what I did not study or discuss with my driving instructor:
- The history of the automobile.
- The theory of combustion engines.
- The common mechanical faults in cars.
- The electrical system of the car.
- The theory of traffic flows.
To this day, I still manage to drive safely without any knowledge on these topics.
In fact, I never expect to learn these topics. I have zero need or interest and they will not help me realize the thing I want and need, which is safe and easy personal mobility.
If the car breaks, I’ll call an expert.
Learning to Code
I started programming without any idea of what coding or software engineering meant.
At home, I messed around with commands in Basic. I messed around with commands in Excel. I modified computer games. And so on. It was fun.
When I started to learn programming and software engineering, it was in university and it was bottom up.
We started with:
- Language theory
- Data types
- Control flow structures
- Data structures
- etc.
When we did get to write code, it was on the command line and plagued with compiler problems, path problems, and a whole host of problems unrelated to actually learning programming.
I hated programming.
Flash-forward a few years. Somehow, I eventually starting working as a professional software engineer on some complex systems that were valued by their users. I was really good at it and I loved it.
Eventually, I did a course that showed how to create graphical user interfaces. And another that showed how to get computers to talk to each other using socket programming. And another on how to get multiple things to run at the same time using threads.
I connected the boring stuff with the thing I really liked: making software that could solve problems, that others could use. I connected it to something that mattered. It was no longer abstract and esoteric.
At least for me, and many developers like me, they taught it wrong. They really did. And it wasted years of time, effort, and results/outcomes that enthusiastic and time-free students like me could dedicate to something they are truly passionate about.
Top-Down Learning
The bottom-up approach is not just a common way for teaching technical topics; it looks like the only way.
At least until you think about how you actually learn.
The designers of university courses, masters of their subject area, are trying to help. They are laying everything out to give you the logical progression through the material that they think will get you to the skills and capabilities that you require (hopefully).
And as I mentioned, it can work for some people.
It does not work for me, and I expect it does not work for you. In fact, very few programmers I’ve met that are really good at their craft came through computer science programs, or if they did, they learned at home, alone, hacking on side projects.
An alternative is the top-down approach.
Flip the conventional approach on its head.
Don’t start with definitions and theory. Instead, start by connecting the subject with the results you want and show how to get results immediately.
Lay out a program that focuses on practicing this process of getting results, going deeper into some areas as needed, but always in the context of the result they require.
It Is Different
It is not the traditional path.
Be careful not to use traditional ways of thinking or comparison if you take this path.
The onus is on you. There is no system to blame. You only fail when you stop.
- It is iterative. Topics are revisited many times with deeper understanding.
- It is imperfect. Results may be poor in the beginning, but improve with practice.
- It requires discovery. The learner must be open to continual learning and discoverery.
- It requires ownership. The learner is responsible for improvement.
- It requires curiosity. The learner must pay attention to what interests them and follow it.
It Is Dangerous
Seriously, I’ve heard “experts” say this many times, saying things like:
You have to know the theory first before you can use this technique, otherwise you cannot use it properly.
I agree that results will be imperfect in the beginning, but improvement and even expertise does not only have to come from theory and fundamentals.
If you believe that a beginner programmer should not be pushing changes to production and deploying them, then surely you must believe that a beginner machine learning practitioner would suffer the same constraints.
Skill must be demonstrated.
Trust must be earned.
This is true regardless of how a skill is acquired.
You’re a Technician
Really!?
This is another “criticism” I’ve seen leveled at this approach to learning.
Exactly. We want to be technicians, using the tools in practice to help people and not be researchers..
You do not need to cover all of the same ground because you have a different learning objective. Although you can circle back and learn anything you like later once you have a context in which to integrate the abstract knowledge.
Developers in industry are not computer scientists; they are engineers. They are proud technicians of the craft.
Efficient, Effective, and a Fun Way to Learn
The benefits vastly outweigh the challenge of learning this way:
- You go straight to the thing you want and start practicing it.
- You have a context for connecting deeper knowledge and even theory.
- You can efficiently sift and filter topics based on your goals in the subject.
It’s faster.
It’s more fun.
And, I bet it makes you much better.
How could you be better?
Because the subject is connected to you emotionally. You have connected it to an outcome or result that matters to you. You are invested. You have demonstrable competence. We all love things we are good at (even if we are a little color blind to how good we are), which drives motivation, enthusiasm, and passion.
An enthusiastic learner will blow straight past the fundamentalist.
Learn Machine Learning
So, how have you approached the subject of machine learning?
Seriously, tell me your approach in the comments below.
- Are you taking a bottom-up university course?
- Are you modeling your learning on such a course?
Or worse:
Are you following a top-down type approach but are riddled with guilt, math envy, and insecurities?
You are not alone; I see this every single day in helping beginners on this website.
To connect the dots for you, I strongly encourage you to study machine learning using the top-down approach.
- Don’t start with precursor math.
- Don’t start with machine learning theory.
- Don’t code every algorithm from scratch.
This can all come later to refine and deepen your understanding once you have connections for this abstract knowledge.
- Start by learning how to work through very simple predictive modeling problems using a fixed framework with free and easy-to-use open source tools.
- Practice on many small projects and slowly increase their complexity.
- Show your work by building a public portfolio.
I have written about this approach many times; see the “Further Reading” section at the end of the post for some solid posts on how to get started with the top-down approach to machine learning.
“Experts” entrenched in universities will say it’s dangerous. Ignore them.
World-class practitioners will tell you it’s the way they learned and continue to learn. Model them.
Remember:
- You learned to read by practicing reading, not by studying language theory.
- You learned to drive by practicing driving, not by studying combustion engines.
- You learned to code by practicing coding, not by studying computability theory.
You can learn machine learning by practicing predictive modeling, not by studying math and theory.
Not only is this the way I learned and continue to practice machine learning, but it has helped tens of thousands of my students (and the many millions of readers of this blog).
Learning Mathematics
Don’t stop there.
A time may come when you want or need to pull back the curtain on the mathematical pillars of machine learning such as linear algebra, calculus, statistics, probability, and so on.
You can use the exact same top-down approach.
Pick a goal or result that matters to you, and use that as a lens, filter, or sift on the topics to study and learn to the depth you need to get that result.
For example, let’s say you pick linear algebra.
A goal might be to grok SVD or PCA. These are methods used in machine learning for data projection, data reduction, and feature selection type tasks.
A top-down approach might be to:
- Implement the method in a high-level library such as scikit-learn and get a result.
- Implement the method in a lower-level library such as NumPy/SciPy and reproduce the result.
- Implement the method directly using matrices and matrix operations in NumPy or Octave.
- Study and explore the matrix arithmetic operations involved.
- Study and explore the matrix decomposition operations involved.
- Study methods for approximating the eigendecomposition of a matrix.
- And so on…
The goal provides the context and you can let your curiosity define the depth of study.
Painted this way, studying math is no different to studying any other topic in programming, machine learning, or other technical subjects.
It’s highly productive, and it’s a lot of fun!
Further Reading
This section provides more resources on the topic if you are looking to go deeper.
Summary
In this post, you discovered the concrete difference between the top-down and bottom-up approaches to learning technical material and why this is the approach that practitioners should and do use to learn machine learning and even related mathematics.
Specifically, you learned:
- The bottom-up approach used in universities to teach technical subjects and the problems with it.
- How people learn to read, drive, and program in a top-down manner and how the top-down approach works.
- The frame of machine learning and even mathematics using the top-down approach to learning and how to start to make rapid progress as a practitioner.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Hi Jason,
From my anecdotal experience I can say that the top down approach suited me more than the bottom up. I have taken a lot of ML & DL courses but I would always either loose interest or get so bogged down by the early complexity that it would deter me from actually learning the fun application’s part. I got way better with ML when I started applying models to real world problems.
Thanks for confirming my bias!
Thanks.
Hey I want to build a ML model to read the PDF and then seggregate based on the diffrence( like creating a diffrent folder and moving those pdf to those folders )
That sounds like a fun project!
Thank you so much for giving insight to the approach of learning, I mostly get borred by the conventional learning and I had two supervisor in my MPhil research work one is a researcher and other is an engineer and I wasted a complete smester of 1 year research in digging the basic of nueral network than I will meet my other supervisor who is an engineer he started things by giving me code of discriminatig the cat and dogs via their image datset and it worked I started my own research topic and finished my research surprisingly in two months
Thanks for sharing, and well done!
Thank you Jason,
I believe this is excellent advice, at least for me. I find I need to oscillate between top down and bottoms up, but here’s the kicker – I can ONLY make myself learn bottoms up when I have motivation which I get from being enthused about doing something exciting like solving a real world problem or making something cool.
I’m looking forward to working through some of your material and also beginning to populate my own learning blog.
Thanks for the clear thinking and inspirational post.
Will
Thanks Will. Hang in there!
I agree, this is the best way to learn machine learning. I skipped all the math and dove into the fun stuff. Now I’m going back and taking a statistics course. I never would have mad it through if I started with statistics
Well done Lissy!
I had the same experience with learning programming in college. Took a class in 2006, to learn C. Had to also use command line and Linux, which made it that much harder. Fast forward to 2012 – learned Java on iTunes U, by moving little robots around and making balls bounce… in an IDE, on my familiar computer. So much more fun. Went on to learn C#, Python, and R in the last few years by doing random little projects that helped me at work or automated something boring at home.
Well done Jeff, thanks for sharing.
Agree and thanks again for putting it in a simpler way.
You’re welcome.
I totally agree with the top-down approach to learning especially in a fast pace field like computing science, data science and machine learning where techniques and technologies transforms faster than you could master the theories behind them. However, I have come to realise my background in computing has helped me in ways it has become really easy to grasp the underlying concepts behind the techniques on-demand.
Thanks for sharing.
I feel like I had an explosion.You articulated really well what I have been doing for a some time already but feeling fear,self-deceit,guilt,paranoia etc.I guess I won’t anymore.
Thank you Jason.
Thanks. Hang in there!
Thank you for writing this. I just discovered your blog and I will be reading everything you’ve written here. 2 months ago, I got super excited about Machine Learning and AI and did the usual googling rounds of “How do I get started in ML” and “Which languages should I learn for ML” and “Prerequisites for ML”.
The results I got from that research shaped my decision of what to study and how to get started – I made a list of all the courses starting from the Ng ML course, got the Multivariable calculus, linear algebra, statistics books, and started teaching myself Python and Calculus. Two Months later? Still reading Python and Calculus books, solving problems, doing exercises (sigh.)
I’m a little bit of a perfectionist (read: fan of bottom-up learning in theory). I mean, plan out everything, take every course, understand every underlying mathematical theory before attempting machine learning courses and EVERYTHING will go smoothly and I will master everything in the shortest amount of time, it’s totally supposed to work right?!
Sometimes, I forget how I learned to read, make music, use a computer, or play a sport. Bottom-up is a beautiful idea, but it only works in a utopian world where I have unlimited motivation every day to master things that don’t directly interest me.
I’m going to change my approach from today (read: throw everything out and play with stuff). I want the excitement back. Thanks again for writing this, Jason.
I’m glad the message helped Tim.
Focus on providing value, being useful. You cannot learn everything, and if you go down that road, almost all of it will not help you to be useful to others (e.g. as a data scientist/ml engineer).
Hang in there.
The famous mathematician Carl Jacobi would probably agree with you. His maxim was “invert, always invert” (German: “man muss immer umkehren”) or WORK BACKWARD.
Another deeply profound post Jason! Thank you.
Thanks Franco, I’m glad it helped.
Thanks for sharing the quote!
Completely agree – great post!
Thanks Patrick!
Thats such an aid.
It seems I have been approaching things the wrong way.
And then I read your article and it tells me why even after studying ml for so long around two years i havent been satisfied.
I took a hybrid approach.
I will get it sorted.
Where should I get started?
I cant start from very basic. I have iteratively left it incomplete.
And cant approach it again.
Suggestions please.
I’m glad it helped.
Focus on learning how to work through problems end to end. This is the one true skill to cultivate.
Start here under “how do i get started”:
https://machinelearningmastery.com/start-here/#getstarted
Very well said. It is a problem in many fields of education.
I remember my first IT class in high school where the teacher explained how a floppy drive is divided into different sections. I mean, seriously? You really need to know this in order to operate a computer? You should have seen the faces of my classmates. I consider myself lucky for being able to see through crap like that. Not everyone can.
I completely agree.
3 months back i started learning ML in bottom up approach. So many i felt bored and i stopped learning again i resumed. My study was intermittent. I felt Top down approach will work for me.
Thank You
Mahesh
*So many Times*
I’m glad to hear that Mahesh, stick with it!
Thank you for the amazing content that you create Jason.
Thanks!
There is some sort of discussion about similarity alignment and also the quirky topic of feedback alignment here:
https://discourse.numenta.org/t/similarity-alignment-a-missing-link-between-structure-function-and-algorithms/3683
I was also practicing html5 and I did a WHT toy example:
http://md2020.eu5.org/wht1.html
Great, how does this relate to the post exactly Sean?
Man, how can I post a gift for you from Brazil, due the insights contained in these texts?
I’m very happy to know that many practioners including you share or have shared time ago the same difficulties that I’ve been crossing right now.
Thanks for your motivational work and technical posts…
You’re giving me a huge help.
Top-Down forever….
Thanks a lot again…
Best regards
Thanks, you’re very kind.
The best help you can give is to share the word about the site with friends and colleagues:
https://machinelearningmastery.com/faq/single-faq/how-can-i-support-you
Strongly agree with this.
I tell my students that it’s like being a race-car driver: you don’t need to be able to engineer the car to be able to drive it.
For a practitioner, it’s almost always best to just learn to use ML tools in a practical way and then “backfill” the mathematics as you need to start fine-tuning models, or as you need to deepen your knowledge.
I think the only major difference in our philosophies is that I strongly suggest that students master basic analysis first: http://www.sharpsightlabs.com/blog/machine-learning-prerequisite-isnt-math/
Thanks for sharing.
I like you post very much!
I think learning a programming language(any language), we always learn it in top-bottom fashion.
Because in order to learn it from bottom-up, this would mean, we need to know how the hardware works, and the OS that operates on the hardware, then the language that written to run in the os, then the compiler, finally the language itself. This is not possible and not efficient for any human being. In order to learn, we don’t need to know how our brain instructs us on how to learn. It’s implicitly Top-down by design.
I agree.
Thank you for your post. I am currently getting a PhD in Informatics in Germany, where we do not have courses. My research area is currently time series analysis with deep learning.
I actually come from physics, and I switched my area of research completely. In the first month of my PhD, I made myself read the free Deep Learning book by Bengio and Goodfellow. While I think it is a good book, I was infinitely bored reading it from morning until evening. And I lost a month, basically. Now I try to use a different approach. For example, I have recently hacked a variational convolutional autoencoder and made it work for my time series dataset. And now I am happy to read the original paper on VAEs. Similarly, to get a better understanding of my models, I am motivated to read books on statistics in my free time. But starting my PhD work with reading a book gave me nearly nothing. In fact, I probably should reread the Deep Learning book, since now, I would certainly get more out of it.
Well done.
How was the skill of your VAE on your time series problem?
I rarely leave replies to articles / blogposts, but the principal and analogies you provide here are spot on. I have a feeling that both your free and paid content will accelerate the learning curve of those that are already motivated to learn, and better yet – may provide the confidence to begin for potential ML learners who feel inadequate due to the perception of complexity, and the often assumed “requirement” of knowledge, both in depth and breadth even to start.
— Start now, Fail often. Nothing motivates more than success learned from prior failure —
Thanks. I’m glad it helped.
I’ve only been learning Machine Learning for about two weeks. I think I have been bouncing back and forth between the bottom up approach to trying to find examples that fit my problem (top down). I am sure the top down approach is the way to go, but my problem isn’t really a discrete problem, it is dealing with a very physical system, whose governing equations are simply to complex to distill into one neat little equation. As a result, I have what seems to me like a supervised system, and it is continuous. I have four inputs, two highly intertwined outputs and just can’t make sense of where to go now. Open to suggestions as I continue to look for examples and methods. Thanks for your site, just found it today and am binge reading… 😉
Perhaps this framework will help clarify the problem:
https://machinelearningmastery.com/how-to-define-your-machine-learning-problem/
Then, perhaps this process will help you work through it:
https://machinelearningmastery.com/start-here/#process
I have advice on how to work through it in a suite of different platforms/libraries here:
https://machinelearningmastery.com/start-here
Let me know how you go Eric.
This is a wonderful article, thank you for spelling it out like that.
I have approached ML and pre reqs from the bottom-up so far, but realize that I could be much farther along if I had approached it differently. I have spent countless hours trying to work though compiler errors or equivalent roadblocks in other subjects and see that a lot of that was wasted time.
This is my first year out of my near finished undergrad program; unfortunately it was not going to be able to deliver its bottom-up promise, and only delaying and indebting me.
I’ve been trying to fill in my gaps with mostly bottom-up approach with studying on my own, but feel buried and lost a lot of the time. There are so many wonderful resources online, I’m trying to balance forward momentum and my tendency to want to know every intricate detail of something before moving on.
Thank you for this great article and the site! I know this will be a treasured resource as I figure out the plan going forward. A top-down learning approach will surely be a better way for me!
I’m glad it helped Grant. Stick with it.
I’m here to help if I can along your journey.
Focus on working problems end to end and delivering results. It is the skeleton that will allow everything else to make sense, or fall away.
if this your way of thinking about learn, then I’m happy that by luck I found you..
Thanks, it’s great to have you here.
Hey Jason,
I think now I can earn anything with your approach and not just programming. 🙂
Thanks.
Thanks for introducing the new approach. I was just thinking of myself of using top down approach and with this article I confirmed… Let’s see how it works…
Great, hang in there!
I like the approach and I have to say everything you’ve mentioned make sense. Personally i hated college for their approach on education and always prefer my “way”, now i see my way as the Top-Down Learning method, and I learn many things that way without even know.
I think it is time to tame Machine Learning then
Thanks, I’m glad it helped.
It really awesome article to start on ML, Thanks Jason
I’m glad it helped.
You learned to drive by practicing driving, not by studying combustion engines.
BUT,
You cannot make a car if you dont know combustion engines.
I totally agree with your arguments and guides. I am also learning university-based machine learning but the top-bottom approach seems much more interesting and more practical. But excelling in this field we really need to know theories.
I understand this article is the motivation for the beginners. And, from your words, I am pushed to learn it in top-bottom approach. I think I can understand the bottom-top approach better if I have a good working knowledge in top-bottom approach.
Thank you for sharing your wisdom.
Thanks for sharing.
Not sure if I agree, you can drive F1s (race cars at the top level) without building them.
Perhaps you can decide if you want to use techniques or develop new techniques (or some mixture). Most business just need someone who can use the techniques and deliver a result, not innovate.
But if you are building cars you really do need to understand what the nuts and bolts do.
It is true that in industry most people will be expected to drive for most of the time, but if you expect to hold a top position for any length of time you will eventually find that the car needs a rebuild and you will then need to understand the nuts and bolts.
You do not need to understand a technique to use a prepackaged version of it, but you do need to understand it if you are going to tweak it, or use it for a different purpose. Top down will let you use an algorithm but not create it.
There is a reason universities teach bottom up – it is not their purpose to teach people how to drive the car it is their job to get students to understand all the facts required to design and build one. Armed with that knowledge and understanding it is up to an employer to teach how to drive their specific car.
If you do not want to understand the car then do not bother going to university instead use excellent resources like this web site to teach you to drive, but do appreciate that they are different tools with different goals and different outcomes.
If you want to win a car race learn to drive if you want to build the best car study engineering. If you want to create a new type of car study science, but if you want to really master any of them you will need to get to grips with all of them.
Top down eventually reaches the bottom, and bottom up eventually reaches the top. Choose the approach that will get you where you want to be soonest.
Personally I find it easier to understand things from first principles and apply them – but I do agree that I appear to be in the minority and a top down context can certainly aid bring a problem into focus and may aid understanding
Sure, but most people are not building cars, nor do they ever need to. My point. We’re driving to the supermarket, not racing in F1. Perhaps you are and that’s fine.
Almost no one needs to develop a new type of neural net or optimization algorithm. Ever.
Just like you can be a java dev without ever having to write a compiler or read bytecodes or write an ide, etc.
But they can fit and tweak models for their entire career and produce a ton of value to business.
Agree with your final point – you MUST find an approach that suits your preferred approach and go all in.
Thanks a lot.
I’m really thrilled by reading this ;I never thought about this – of changing current education system.But when I read this I felt like this is all around me … I wasn’t able to understand it.
And I’m in my early stages of Machine Learning learning process. Sure this insight will help me
I’m glad it helped.
Hang in there, you’re not alone. There are millions of us – getting things done and delivering results.
Thanks for sharing your valuable points sir.
I hope it helped.
That was truly inspirational for someone like me who is in the early stages of learning Machine Learning, and feel like I’m over my head and drowning in jargon and information overload.
Thank you!
Thanks Anthony, I’m glad it helped!
Hi, your post was extremely helpful as I am just in the beginning stage. One question which I would like to ask is like you mentioned about a child learning how to read, a person learning how to drive in a top-down fashion, we also train machines in a similar fashion right?
Yes, from example.
Thanks Sir Jason.
Indeed perfect Top down approach which should admire by every educational institutes.
Thanks for being such a visionary coach.
Thanks.
That’s very inspirational. Thank you!
Thanks, I’m glad it helped.
Faced a similar problem when I was introduced to programming in C in 10th grade. We were made to learn functions, pointers, data structures (literally cried my lungs out for this) and so on. I believed coding is something I would never want to do in my life.
Got hired by a company having a team focusing on machine learning and artificial neural networks. I learnt python from scratch and wrote codes myself with the help of my manager.
I can happily say that I have never found coding so liberating before. Whatever I learned, I did it through extensive google-ing, experimenting and playing around with bits of code.
I could totally relate to your blog. Thanks so much for making me feel like I wasn’t the only one to hate programming in my initial years.
Thanks for sharing.
Yes, you/we are not alone. I think it is a lot more common than is discussed.
What a great and inspiring post, thanks for writing this!
I found this site searching for some mathematics term that I’d never heard of, which I read in a ML book (and this was just the introduction chapter!).
I’d just about given up on learning ML having attempted a bottom-up style, starting with some maths courses, only to discover that I didn’t even have enough knowledge to understand those so then I need to take even simpler maths courses… I was beginning to think I’d need to go back to school level basics before I could get anywhere.
I don’t know why I approached it like this though, it’s certainly not how I learned to program nor how I would attempt to learn anything else, I guess there’s something about ML that feels a bit abstract and intimidating.
Anyway I’m going to change my approach now and jump in and try to do something, I can work back from there if needed.
Thanks Pete, I’m so glad the post helped.
Stick with it, I’m here to help if have questions about other ML terms!
i´ll start from here…
Thanks.
Jason, you’re the man!
I agree with you totally.
I worked with some of the most amazing programmers in IT. This was back in 2002 and these guys had created an application for the orthotics/prosthetics industry.
They became the Microsoft of that industry. These guys created the application to solve a particular problem. None of them had taken any formal programming classes, and none of them had even started college. They just liked programming and would even create projects in their spare time. (They got their college degrees years after being professional programmers).
I loved programming but I was awful at it. For every 25-50 lines of code I created these guys could do the same with 5!
I thought that maybe I should take classes. I did take classes but it still wasn’t clicking.
I think a part of the issue was that I wanted to solve a problem and there were already solutions. I’d get challenged to write a particular program, but then I’d think, “What’s the point? Someone already wrote one.”
It wasn’t personal to me.
It wasn’t until I got into SharePoint and SQL that things changed. I had no choice but to create queries and develop SharePoint apps specifically for my company at that time. I even ended up recoding parts of Procmail to process different file types back when I was a Linux admin.
It took me a heck of a long time, and much of my code was crude (and even laughable at times) but I liked it and it was ‘me’.
As time went on the theory I had learned started to click because my mind was relating it to something practical.
I like writing stories quite a bit. Ever since I was a kid I’d write some adventure story or other. People liked my work but it needed “polish”. So I went and studied how to write novels, learned the rules of grammar, etc. But I started by writing stories that I liked and then connected to the theory later.
In my opinion, a true sign of intelligence is how well a person can take a complicated topic and make it palatable.
You’ve done that!
I’ve always been interested in machine learning, but had no idea where to start. Thanks for your website and all your efforts. It’s pointing me in the right direction!
Fantastic Nick, thank you for sharing, it’s greatly appreciated.
None of us are alone in this thing.
First time going throug ML. I’m instructed by my final year supervisor to machine learning. But I’m curios to ML. I will be happy will all help/material/tools for ML. As it will be my final year project.
I find this post extreme helpful. I do regard button-up approach and top-down approach. I’m novice to machine Learning. I’m curious to it. But pls help me with useful tools
I noticed you replied to a lot of the comments that most people will not need to create a new neural network architecture. That’s probably correct, but there are quite a few of us out there who like to work close to the metal, even if we don’t build a new car. Even in your analogy, I’m not comfortable merely driving the car. I have to know how the car works. I have to know the history. I don’t feel like I truly understand how to drive unless I know what’s inside the black or gray boxes.
Practically speaking, this manifests (for me, at least) as a dearth of material connecting the top to the bottom. For speed, I have to start top-down, but eventually, I get to a place where all the top-down articles stop, or handwave something that’s really important. You search for more info, but all you can find are esoteric articles written for people who started from the bottom up, filled with jargon that only makes sense if you’ve steeped yourself in the bottom-up universe. In other words, there seems to be a limit on how far down you can go from the top in the top-down literature.
Fair enough, follow the approach that best suits your learning style.
Perhaps. Once you learn the nomenclature for a field, diving deeper gets easier. It takes practice – e.g. learning what to focus on.
Thanks you Jason for such a great and amazing guidance.
I just started with ML, tried different coerces and approach to study and understand the ML concepts but I like the approach suggested by you.
I will follow it.
Thank you
Thanks.
Fantastic post. I’m an archaeologist who used to struggle with maths… and because of this, I wouldn’t try to see how it could be applied. When I began hearing about the success of Machine Learning algorithms in other fields of science, such as medicine, it seriously caught my attention…
I’ve been familiar with R for many years. In archaeology, many people frequently use R to do basic statistics. I always found using R relatively fun… because after hours of errors, the moment you get the code to actually work It always feels like you’ve achieved something. This got me into more advanced statistics in R and eventually Machine Learning. Through this you can probably say I followed what you call the Top Down approach. I began to find algorithms that worked, applied them to my data sets, then tried to find alternatives and other ways of achieving the best results possible. I eventually began playing around with Neural Networks and Deep Learning and was amazed at the results. I have to say a great deal of my success was thanks to Machine Learning Mastery books among other.
After having found the success of applying these algorithms, I now feel a much more experienced programmer, especially in R, and have spent the last few months developing algorithms in Python. I’ve also began reading in to the fundamentals of Machine Learning theory, and now I think I can even say I quite enjoy maths and use it on a daily basis.
I think the Top Down approach for many scientists from other fields interested in implementing these algorithms is the best way to begin learning about Data Science. For most archaeologists, they used to think what I do is science fiction… at least until they begin seeing the results I’ve been producing for the better part of a year now.
Anyway, I really appreciate these posts and the books. I follow this website very closely and with it was able to develop my Master’s Thesis. I’ve just started research for my PhD research and can definitely say I’ll probably be revising these posts very closely over the next couple of years – highly recommendable
Many thanks Jason and keep up the good work!
Thanks for sharing Lloyd, a wonderful and encouraging story.
I wish you the best of luck with your PhD!
Great article! I just want to share how I started learning ML, on 14th July 2019 I joined a meetup on TensorFlow 2.0 at Google Developer space without any prior knowledge of the subject. After the talk, we had an opportunity to play with TensorFlow and deploy it on Coral to perform object recognition. I truly enjoyed the event and I wanted to know more. I found the course TensorFlow in Practice in Coursera and for the past few days I’ve been learning it using the top-to-bottom approach. While doing my research I stumble upon your website and I’m glad I found it. Thank you. 🙂
Thanks for sharing, and well done!
Pls I need ur help. It is my final year project area. Need ur help. Please share inportant hint on ML tome @Abubakarashir80@gmail.com
You can get started here:
https://machinelearningmastery.com/start-here/
I am so happy I stumbled upon your website and found the top down approach, which is exactly the way I love to study, pick info on the go and for practically applicable purposes. I’m just beginning here after doing the course by Andrew Ng
Thanks. It’s great to have you here as part of the ML Mastery community.
do you write all posts yourselt, it is awesome.
i am reading book of “Hands-on Machine Learning with Scikit-Learn and Tensorflow”, and trying to fix some basic competitions on Kaggle.
Good luck!
Yes I do!
Hi Jason,
I am a huge fan of your website. What I love the most is your approach to ML. In a subject that can be deeply technical/challenging, you are putting the person and the learner’s mindset ahead of the technical matter. Recognizing that people learn better in the context of achieving a result or solving a problem is key. The simple Iris example was a major confidence booster in an area where most people talk about complexities/mathematical modeling first. Congratulations on the making the positive impact you are and looking forward to learning more from your content.
Sumitha
Thanks Sumitha, I appreciate your support!
Like your thinking and your expression.
Ralph
Thanks Ralph!
I’m currently taking a bottom-up university math course due to guilt, math envy and insecurities from following a top-down approach. I’m noticing that it’s really hard to relate what I’m learning to what I use (so far), and this makes it hard to stay motivated. Time to revisit my approach, thanks for opening my eyes!
Yes, they are worlds apart.
Like the theory of computation and programming.
Preach Jason!
I must admit I have never found an article that reflected “exactly” how my journey into data science has gone thus far. I’ve also heard one of my favorite MOOC instructors state that “you can revisit and refine your understanding of the details at anytime; however, it’s far more important to have a framework of knowledge” (and I’m paraphrasing probably poorly).
That being said, I will add one more reason why the bottom-up, university regurgitation SUCKS! By only expanding/developing on the work of few who originally paved the way (Newtonian physics for example), you completely disregard fundamental, potentially revolutionary pathways that allow for a larger diversity of thought/ideas. Unfortunately, the authors of these original works are the gatekeepers of new scientific publications, and let’s be honest, they are some prideful folks. I think I’ve overstayed my welcome on this reply; however, I want to thank you Jason for your candor and I too invite everyone to try the top-down approach! You’re not a robot!
Thanks Jake!
I’m so happy the post rang true for you, thanks for sharing your experience.
I see so many “practitioners” studying to become “academics”, like lemmings walking off a cliff. Such a waste of potential.
Hello, I was advised by my younger brother to read your point of view and how to understand everything in machine learning. The thing is, I’m in doctoral student and now I’m stuck at a certain stage of my PhD research, and I like your approach. I didn’t realize it before, but in life I moved in way you writing about. Theory has never been my strong side, I prefer a practical approach to things. Thank you, hope me this will help.
Thanks, I’m happy it helped!
Hang in there and good luck with your research!
>>Are you following a top-down type approach but are riddled with guilt, math envy, and insecurities?
When I went through those words, I was like is he in the same room as me!? That’s the exact feeling I had when I had just thought about taking up top-down approach. Thanks a lot for brushing away these feelings in the subsequent section. I still have a lingering feeling but I guess once I take the first few steps, things will get going and moving forward.
Hang in there!
I’m so glad to find your website, this gave me motivation to get into machine learning after so much time thinking about on how to start
Thanks!
You have PhD in applied Psychology. Lol 🙂 you know how to read people’s minds, you can profile people at first look. You have a balanced mindset and a very level-headed person. I can read you too with the way you write. probably more than 50% of your brain is being utilized at any single time. You are on your way to become a mutant. The AI machine itself should study you.
Seriously, you know how people’s mind work.
On the other hand, I think that AI has a long long way to go. But these contributions you are making help this world advance further.
My thoughts on ML:
Machine learning also cannot involve learning through senses like what a human being can do. We learn not only by visual perception but through touch, feel, hear, smell, taste not to mention human perception and imagination etc. You teach an “infant” AI by mimicking how you teach a toddler going to childhood to adulthood then AI can transcend what it terms as “training” to what I call as “experience” – just like what humans do we learn through experiences in life. My thinking is that if we teach an AI the way we teach a toddler and as the toddler grows he will have the same level of knowledge as an adult human being – only way way so much better.
Thanks, and thanks for sharing your thoughts.
Thanks Jason,
During Machine Learning courses I was feeling bad just because I did not understand the whole theory. Now, I realized that it is not necessary to get all logic behind math and stat, there is still some time to go back and think about them to understand better.
Exactly!
I was not sure that I had started learning machine learning. However, after reading this blog I know I am. I recently jumped on Kaggle and went through the intro to machine learning and intro to Python. I am working on the Titanic problem now. I did not need 4 years of college to get to this point.
Well done!
I have many cousins who migrated to middle east from India. They learnt arabic without any proper classes, only by practicing with people.
Impressive, thanks for sharing!
Thanks a lot, it is really helpful, however, I’ve got a question. Now that I have very basic knowledge of Python and ML, how can I get involved in small projects? Where can I find them and join as a part of team or as an individual project? I don’t have a job in this field! I’m just sitting home learning it.
Regards
You can do it yourself, e.g. practice on small standard datasets:
https://machinelearningmastery.com/practice-machine-learning-with-small-in-memory-datasets-from-the-uci-machine-learning-repository/
Dude. Thank you ???????? I am in the midst of combining a nine-year career in emergency nursing with a nascent understanding of Python (all things coding & programming really) with the intention of making a new career. My journey and curiosity led me to machine learning. I’m not sure what that means but your manner of gently explaining MCMC gave me the confidence to explore more machine learning with you. Let’s do this!!!
You’re welcome.
Hi Jason, thank you for the wonderful post. I started learning in top down manner and it is quite successful. I need a small suggestion from you, i am going to start my Ph.D work soon and i would like to work in object detection / classification of images using Deep Learning. Can you help me what are the recent research problems in this area. or any other area in which there is a scope to work on.
Thank you in advance
That’s great to hear!
Here’s my advice on selecting a research topic:
https://machinelearningmastery.com/faq/single-faq/what-research-topic-should-i-work-on
that was fantastic jason.i started a top-down approach on programming 4 years ago, that was freaking fantastic.
Thanks!
i started a top-down approach on programming 4 years ago, that was freaking fantastic.
and lately i was about to make a mistake and start learning mathematics like a pro for machine learning, i didnt know about this approches, thanks for insight man, that was awsome 🙂
You’re welcome!
Grazie Jason, per me sei stato chiaro esplicito e di grande aiuto, specialmente dal punto di vista psicologico.
Non dimentichero’ mai cio che hai suggerito, facendolo in modo semplice e forte.
I miei complimenti e ringraziamenti. Sei un grande .
You’re very welcome, I’m happy it helps!
The peptalk i needed.
I have an idea for my bachelor thesis but always made myself believe I had to conquer it from the botttom even tho I got pretty far by learning to code on projects.
Thank you very much !
You’re welcome, you’re not alone.
This is brilliant thanks a lot Jason ! This is exactly the reason university made me feel lost and I think your top-down really helps motivate people.
You’re welcome, I’m happy it helps!
I have decided to adopt a top-down approach and try all the project that you have in your curriculum. It is way more exciting to start with projects. One thing I think is also good to help learning is to have a fixed cadence of learning. This is how I learned SEO, and how I will learn Machine learning. One article at the time. Thanks Jason
You’re welcome!
Fantastic article. I’m a top-down learner ordinarily but the world of programming seemed quite daunting for years (honestly even the jargon was like learning a second language, taking one look at the Python documentation gave me nightmares), so I opted to learn from the bottom-up. It isn’t my style and there aren’t many good tutors online, even with the large universities so finding a way to learn was difficult. Then I endured everything you listed; maths envy, insecurity, you name it. The biggest problem was feeling as if I *had* to learn like everybody else. I’m not much of a conformist and do pretty much everything differently to most, and learning like most proved nigh-on impossible.
I started by looking at ML, finding I ‘got it’ pretty rapidly but decided I wanted a decent foundation, did the P4E course through the University of Michigan, almost every Python challenge online, a few HackerRank challenges, some LeetCode challenges, went through about 500 cycles of despair wondering what else I thought I had to do, which programming languages I thought I should pick up etc., learning maths, doing AI courses, then I went onto Kaggle and realised there wasn’t really anything to worry about! I still don’t like coding much but I love ML, it feels like a good place too be, everything seems to work logically, it isn’t OOP (coding trading algorithms is cripplingly boring), and the libraries are fascinating. I’m glad I got the foundation but I’m kicking myself for wasting weeks / months on HackerRank and LeetCode and stressing over what to do next when I should have gone straight to ML and picked up what I needed to learn this way. Anyway, it can’t hurt to have learned all that. And, great site by the way! I’ve looked at a metric s**t-ton of course material online and I think you have the best teaching style and material i’ve seen yet. Glad to have found your site.
Thanks!
Thank you for sharing your story!
I have not started yet, but stopped many a times thinking how the hell am I supposed to catch up on so much of literature. Then it occurred – top down is the only way ahead and then I cam across your article. Thanks for writing this out, increases my willingness to learn.
Happy to hear that. Hang in there!
Its so good to get stumbled upon in your website as an ambitious data scientist…Truely am a fan of it…and it encourages me to learn.
Thank you for the great effort and the article
Thank you. Hope you enjoy it.
you are a great teacher ,
I like your manner
I wish you success.
hossein
I liked the way you narrated the difference between bottom up and top down approach. While I have learnt many new things with top down approach, this narration makes it clear on how things can be learnt quickly. Thanks for putting this together.
Great feedback Kiran!
Great post!! Thank you Sir for sharing your knowledge.
My knowledge is getting enriched after reading your blogs.
Great feedback Medini!
I never attempted programming before my master’s and I had to do a research on AI, I wish I found this earlier, the painful thing is I understand what needs to be done but never confident of trying it out on my own, I am also scared of getting errors, I don’t know where to start when they occur
Thank you for the feedback Jonpaul! The following is a great starting point for your machine learning journey:
https://machinelearningmastery.com/start-here/
WOW that’s the coolest, the most helpful post, at least for me. Thanks a lot, Jason! Really appreciate it. In first semester at university I had a course on C programming, but I failed that subject) I thought I’m stupid and never gonna touch programming. But in my 4th year, one course covered intro to python. It was fun and then I took 2 courses of python in Coursera and got my first job due to these courses. At that job I had to look back to C again and I was shocked how easy and understandable it was) I didn’t realise it was top-down approach! I also failed some courses that seemed interesting at first, but got so boring and tough when I studied it. One of them was Signals and Systems. When it started topics like Convolution, Fourier Series, I got confused. Now, even late, I now how to learn such subjects!
Thank you Ali for your support and feedback! The following location is a great starting point for your machine learning journey:
https://machinelearningmastery.com/learning-curves-for-diagnosing-machine-learning-model-performance/
What can a doctor say? completely agree! Top-down approach is the way. I was taught medicine bottom-up. Out of medical school, still unable to make clinical diagnosis when I see a patient. What got me there in the end? Top-down approach; see the patient, what is his problem? what does he need? go back to books, pick what you want to study-make a diagnosis, go back again, what is the treatment? That is the way I finally learnt medicine. Knowing that, when I started learning python/ML this year, knew I need to go with top-down approach. When I read this post, how surprisingly similar thinking we both have!
Great feedback Ramesh! We greatly appreciate it!
The vast majority of junior jobs in the industry don’t involve major risks because you’re a junior. There are some jobs, such as Biostatistics, that require a very solid and good foundation in mathematics and statistics, but these jobs are definitely not junior roles. Or, on the other hand, there are roles such as Big Data Engineer, which can cause the company to lose tens of thousands of dollars with a wrong mistake, but these roles generally require 5-6 years of experience in Software Engineering. In summary, the article author is absolutely right, because they will not trust you as a junior in very risky projects for many years anyway (and this is how it should be)
Thank you Mach for your feedback and contribution to our discussion!
It’s also great that ML offers the opportunity for top-down learning. You don’t have such a chance when you want to work in a field such as microprocessors.
Hi Mach…You may find the following of interest:
https://www.renesas.com/us/en/document/gde/ultimate-guide-machine-learning-embedded-systems#:~:text=Machine%20Learning%20in%20Embedded%20Systems,compared%20to%20our%20traditional%20computers.
Thanks a ton, Dr. Jason for the invaluable insights illustrating and presenting various approaches to get motivated for the machine learning journey. I can’t wait to start!
You are very welcome Hansel! We appreciate your feedback and support!