You’re Doing it Wrong. Why Machine Learning Does Not Have to Be So Hard

Technical topics like mathematics, physics, and even computer science are taught using a bottom-up approach.

This approach involves laying out the topics in an area of study in a logical way with a natural progression in complexity and capability.

The problem is, humans are not robots executing a learning program. We require motivation, excitement, and most importantly, a connection of the topic to tangible results.

Useful skills we use every day like reading, driving, and programming were not learned this way and were in fact learned using an inverted top-down approach. This top-down approach can be used to learn technical subjects directly such as machine learning, which can make you a lot more productive a lot sooner, and be a lot of fun.

In this post, you will discover the concrete difference between the top-down and bottom-up approaches to learning technical material and why this is the approach that practitioners should use to learn machine learning and even related mathematics.

After reading this post, you will know:

  • The bottom-up approach used in universities to teach technical subjects and the problems with it.
  • How people learn to read, drive, and program in a top-down manner and how the top-down approach works.
  • The frame of machine learning and even mathematics using the top-down approach to learning and how to start to make rapid progress as a practitioner.

Let’s get started.

You're Doing it Wrong. Why Machine Learning Does Not Have to Be So Hard

You’re Doing it Wrong. Why Machine Learning Does Not Have to Be So Hard
Photo by popofatticus, some rights reserved.

Overview

This is an important blog post, because I think it can really help to shake you out of the bottom-up, university-style way of learning machine learning.

This post is divided into seven parts; they are:

  1. Bottom-Up Learning
  2. Learning to Read
  3. Learning to Drive
  4. Learning to Code
  5. Top-Down Learning
  6. Learn Machine Learning
  7. Learning Mathematics

Bottom-Up Learning

Take a field of study, such as mathematics.

There is a logical way to lay out the topics in mathematics that build on each other and lead through a natural progression in skills, capability, and understanding.

The problem is, this logical progression might only make sense to those who are already on the other side and can intuit the relationships between the topics.

Most of school is built around this bottom-up natural progression through material. A host of technical and scientific fields of study are taught this way.

Think back to high-school or undergraduate studies and the fundamental fields you may have worked through: examples such as:

  • Mathematics, as mentioned.
  • Biology.
  • Chemistry.
  • Physics.
  • Computer Science.

Think about how the material was laid out, week-by-week, semester-by-semester, year-by-year. Bottom-up, logical progression.

The problem is, the logical progression through the material may not be the best way to learn the material in order to be productive.

We are not robots executing a learning program. We are emotional humans that need motivation, interest, attention, encouragement, and results.

You can learn technical subjects from the bottom-up, and a small percentage of people do prefer things this way, but it is not the only way.

Now, if you have completed a technical subject, think back to how to you actually learned it. I bet it was not bottom-up.

Learning to Read

Think back; how did you learn to read?

My son is starting to read. Without thinking too much, here are the general techniques he’s using (really the school and us as parents):

  • Start by being read to in order to generate interest and show benefits.
  • Get the alphabet down and making the right sounds.
  • Memorize the most frequent words, their sounds, and how to spell them.
  • Learn the “spell-out-the-word” heuristic to deal with unknown words.
  • Read through books with supervision.
  • Read through books without supervision.

It is important that he continually knows why reading is important, connected to very tangible things he wants to do, like:

  • Read captions on TV shows.
  • Read stories on topics he loves, like Star Wars.
  • Read signs and menus when we are out and about.
  • So on…

It is also important that he gets results that he can track and in which he can see improvement.

  • Larger vocabulary.
  • Smoother reading style
  • Books of increasing complexity.

Here’s how he did not learn to read:

  • Definitions of word types (verbs, nouns, adverbs, etc.)
  • Rules of grammar.
  • Rules of punctuation.
  • Theory of human languages.

Learning to Drive

Do you drive?

It’s cool if you don’t, but most adults do out of necessity. Society and city design is built around personal mobility.

How did you learn to drive?

I remember some written tests and maybe a test on a computer. I have no memory of studying for them, though I very likely did. Here’s what I do remember.

I remember hiring a driving instructor and doing driving lessons. Every single lesson was practical, in the car, practicing the skill I was required to master, driving the vehicle in traffic.

Here’s what I did not study or discuss with my driving instructor:

  • The history of the automobile.
  • The theory of combustion engines.
  • The common mechanical faults in cars.
  • The electrical system of the car.
  • The theory of traffic flows.

To this day, I still manage to drive safely without any knowledge on these topics.

In fact, I never expect to learn these topics. I have zero need or interest and they will not help me realize the thing I want and need, which is safe and easy personal mobility.

If the car breaks, I’ll call an expert.

Learning to Code

I started programming without any idea of what coding or software engineering meant.

At home, I messed around with commands in Basic. I messed around with commands in Excel. I modified computer games. And so on. It was fun.

When I started to learn programming and software engineering, it was in university and it was bottom up.

We started with:

  • Language theory
  • Data types
  • Control flow structures
  • Data structures
  • etc.

When we did get to write code, it was on the command line and plagued with compiler problems, path problems, and a whole host of problems unrelated to actually learning programming.

I hated programming.

Flash-forward a few years. Somehow, I eventually starting working as a professional software engineer on some complex systems that were valued by their users. I was really good at it and I loved it.

Eventually, I did a course that showed how to create graphical user interfaces. And another that showed how to get computers to talk to each other using socket programming. And another on how to get multiple things to run at the same time using threads.

I connected the boring stuff with the thing I really liked: making software that could solve problems, that others could use. I connected it to something that mattered. It was no longer abstract and esoteric.

At least for me, and many developers like me, they taught it wrong. They really did. And it wasted years of time, effort, and results/outcomes that enthusiastic and time-free students like me could dedicate to something they are truly passionate about.

Top-Down Learning

The bottom-up approach is not just a common way for teaching technical topics; it looks like the only way.

At least until you think about how you actually learn.

The designers of university courses, masters of their subject area, are trying to help. They are laying everything out to give you the logical progression through the material that they think will get you to the skills and capabilities that you require (hopefully).

And as I mentioned, it can work for some people.

It does not work for me, and I expect it does not work for you. In fact, very few programmers I’ve met that are really good at their craft came through computer science programs, or if they did, they learned at home, alone, hacking on side projects.

An alternative is the top-down approach.

Flip the conventional approach on its head.

Don’t start with definitions and theory. Instead, start by connecting the subject with the results you want and show how to get results immediately.

Lay out a program that focuses on practicing this process of getting results, going deeper into some areas as needed, but always in the context of the result they require.

It Is Different

It is not the traditional path.

Be careful not to use traditional ways of thinking or comparison if you take this path.

The onus is on you. There is no system to blame. You only fail when you stop.

  • It is iterative. Topics are revisited many times with deeper understanding.
  • It is imperfect. Results may be poor in the beginning, but improve with practice.
  • It requires discovery. The learner must be open to continual learning and discoverery.
  • It requires ownership. The learner is responsible for improvement.
  • It requires curiosity. The learner must pay attention to what interests them and follow it.

It Is Dangerous

Seriously, I’ve heard “experts” say this many times, saying things like:

You have to know the theory first before you can use this technique, otherwise you cannot use it properly.

I agree that results will be imperfect in the beginning, but improvement and even expertise does not only have to come from theory and fundamentals.

If you believe that a beginner programmer should not be pushing changes to production and deploying them, then surely you must believe that a beginner machine learning practitioner would suffer the same constraints.

Skill must be demonstrated.

Trust must be earned.

This is true regardless of how a skill is acquired.

You’re a Technician

Really!?

This is another “criticism” I’ve seen leveled at this approach to learning.

Exactly. We want to be technicians, using the tools in practice to help people and not be researchers..

You do not need to cover all of the same ground because you have a different learning objective. Although you can circle back and learn anything you like later once you have a context in which to integrate the abstract knowledge.

Developers in industry are not computer scientists; they are engineers. They are proud technicians of the craft.

Efficient, Effective, and a Fun Way to Learn

The benefits vastly outweigh the challenge of learning this way:

  • You go straight to the thing you want and start practicing it.
  • You have a context for connecting deeper knowledge and even theory.
  • You can efficiently sift and filter topics based on your goals in the subject.

It’s faster.

It’s more fun.

And, I bet it makes you much better.

How could you be better?

Because the subject is connected to you emotionally. You have connected it to an outcome or result that matters to you. You are invested. You have demonstrable competence. We all love things we are good at (even if we are a little color blind to how good we are), which drives motivation, enthusiasm, and passion.

An enthusiastic learner will blow straight past the fundamentalist.

Learn Machine Learning

So, how have you approached the subject of machine learning?

Seriously, tell me your approach in the comments below.

  • Are you taking a bottom-up university course?
  • Are you modeling your learning on such a course?

Or worse:

Are you following a top-down type approach but are riddled with guilt, math envy, and insecurities?

You are not alone; I see this every single day in helping beginners on this website.

To connect the dots for you, I strongly encourage you to study machine learning using the top-down approach.

  • Don’t start with precursor math.
  • Don’t start with machine learning theory.
  • Don’t code every algorithm from scratch.

This can all come later to refine and deepen your understanding once you have connections for this abstract knowledge.

  1. Start by learning how to work through very simple predictive modeling problems using a fixed framework with free and easy-to-use open source tools.
  2. Practice on many small projects and slowly increase their complexity.
  3.  Show your work by building a public portfolio.

I have written about this approach many times; see the “Further Reading” section at the end of the post for some solid posts on how to get started with the top-down approach to machine learning.

“Experts” entrenched in universities will say it’s dangerous. Ignore them.

World-class practitioners will tell you it’s the way they learned and continue to learn. Model them.

Remember:

  • You learned to read by practicing reading, not by studying language theory.
  • You learned to drive by practicing driving, not by studying combustion engines.
  • You learned to code by practicing coding, not by studying computability theory.

You can learn machine learning by practicing predictive modeling, not by studying math and theory.

Not only is this the way I learned and continue to practice machine learning, but it has helped tens of thousands of my students (and the many millions of readers of this blog).

Learning Mathematics

Don’t stop there.

A time may come when you want or need to pull back the curtain on the mathematical pillars of machine learning such as linear algebra, calculus, statistics, probability, and so on.

You can use the exact same top-down approach.

Pick a goal or result that matters to you, and use that as a lens, filter, or sift on the topics to study and learn to the depth you need to get that result.

For example, let’s say you pick linear algebra.

A goal might be to grok SVD or PCA. These are methods used in machine learning for data projection, data reduction, and feature selection type tasks.

A top-down approach might be to:

  1. Implement the method in a high-level library such as scikit-learn and get a result.
  2. Implement the method in a lower-level library such as NumPy/SciPy and reproduce the result.
  3. Implement the method directly using matrices and matrix operations in NumPy or Octave.
  4. Study and explore the matrix arithmetic operations involved.
  5. Study and explore the matrix decomposition operations involved.
  6. Study methods for approximating the eigendecomposition of a matrix.
  7. And so on…

The goal provides the context and you can let your curiosity define the depth of study.

Painted this way, studying math is no different to studying any other topic in programming, machine learning, or other technical subjects.

It’s highly productive, and it’s a lot of fun!

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Summary

In this post, you discovered the concrete difference between the top-down and bottom-up approaches to learning technical material and why this is the approach that practitioners should and do use to learn machine learning and even related mathematics.

Specifically, you learned:

  • The bottom-up approach used in universities to teach technical subjects and the problems with it.
  • How people learn to read, drive, and program in a top-down manner and how the top-down approach works.
  • The frame of machine learning and even mathematics using the top-down approach to learning and how to start to make rapid progress as a practitioner.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

55 Responses to You’re Doing it Wrong. Why Machine Learning Does Not Have to Be So Hard

  1. The c0d3R March 28, 2018 at 6:05 am #

    Hi Jason,

    From my anecdotal experience I can say that the top down approach suited me more than the bottom up. I have taken a lot of ML & DL courses but I would always either loose interest or get so bogged down by the early complexity that it would deter me from actually learning the fun application’s part. I got way better with ML when I started applying models to real world problems.

    Thanks for confirming my bias!

  2. Will Bidstrup March 28, 2018 at 6:24 am #

    Thank you Jason,

    I believe this is excellent advice, at least for me. I find I need to oscillate between top down and bottoms up, but here’s the kicker – I can ONLY make myself learn bottoms up when I have motivation which I get from being enthused about doing something exciting like solving a real world problem or making something cool.

    I’m looking forward to working through some of your material and also beginning to populate my own learning blog.

    Thanks for the clear thinking and inspirational post.

    Will

  3. Lissy March 28, 2018 at 11:57 am #

    I agree, this is the best way to learn machine learning. I skipped all the math and dove into the fun stuff. Now I’m going back and taking a statistics course. I never would have mad it through if I started with statistics

  4. Jeff March 28, 2018 at 12:41 pm #

    I had the same experience with learning programming in college. Took a class in 2006, to learn C. Had to also use command line and Linux, which made it that much harder. Fast forward to 2012 – learned Java on iTunes U, by moving little robots around and making balls bounce… in an IDE, on my familiar computer. So much more fun. Went on to learn C#, Python, and R in the last few years by doing random little projects that helped me at work or automated something boring at home.

  5. Raj March 28, 2018 at 6:54 pm #

    Agree and thanks again for putting it in a simpler way.

  6. Muyi Ibidun March 28, 2018 at 10:06 pm #

    I totally agree with the top-down approach to learning especially in a fast pace field like computing science, data science and machine learning where techniques and technologies transforms faster than you could master the theories behind them. However, I have come to realise my background in computing has helped me in ways it has become really easy to grasp the underlying concepts behind the techniques on-demand.

  7. Dhrumil March 29, 2018 at 12:36 pm #

    I feel like I had an explosion.You articulated really well what I have been doing for a some time already but feeling fear,self-deceit,guilt,paranoia etc.I guess I won’t anymore.

    Thank you Jason.

  8. Tim March 29, 2018 at 3:38 pm #

    Thank you for writing this. I just discovered your blog and I will be reading everything you’ve written here. 2 months ago, I got super excited about Machine Learning and AI and did the usual googling rounds of “How do I get started in ML” and “Which languages should I learn for ML” and “Prerequisites for ML”.

    The results I got from that research shaped my decision of what to study and how to get started – I made a list of all the courses starting from the Ng ML course, got the Multivariable calculus, linear algebra, statistics books, and started teaching myself Python and Calculus. Two Months later? Still reading Python and Calculus books, solving problems, doing exercises (sigh.)

    I’m a little bit of a perfectionist (read: fan of bottom-up learning in theory). I mean, plan out everything, take every course, understand every underlying mathematical theory before attempting machine learning courses and EVERYTHING will go smoothly and I will master everything in the shortest amount of time, it’s totally supposed to work right?!

    Sometimes, I forget how I learned to read, make music, use a computer, or play a sport. Bottom-up is a beautiful idea, but it only works in a utopian world where I have unlimited motivation every day to master things that don’t directly interest me.

    I’m going to change my approach from today (read: throw everything out and play with stuff). I want the excitement back. Thanks again for writing this, Jason.

    • Jason Brownlee March 30, 2018 at 6:31 am #

      I’m glad the message helped Tim.

      Focus on providing value, being useful. You cannot learn everything, and if you go down that road, almost all of it will not help you to be useful to others (e.g. as a data scientist/ml engineer).

      Hang in there.

  9. Franco March 29, 2018 at 7:45 pm #

    The famous mathematician Carl Jacobi would probably agree with you. His maxim was “invert, always invert” (German: “man muss immer umkehren”) or WORK BACKWARD.

    Another deeply profound post Jason! Thank you.

    • Jason Brownlee March 30, 2018 at 6:36 am #

      Thanks Franco, I’m glad it helped.

      Thanks for sharing the quote!

  10. Patrick March 29, 2018 at 10:18 pm #

    Completely agree – great post!

  11. Doi April 6, 2018 at 8:24 pm #

    Thats such an aid.
    It seems I have been approaching things the wrong way.
    And then I read your article and it tells me why even after studying ml for so long around two years i havent been satisfied.
    I took a hybrid approach.

    I will get it sorted.
    Where should I get started?
    I cant start from very basic. I have iteratively left it incomplete.
    And cant approach it again.

    Suggestions please.

  12. BitBarrel April 11, 2018 at 7:36 pm #

    Very well said. It is a problem in many fields of education.

    I remember my first IT class in high school where the teacher explained how a floppy drive is divided into different sections. I mean, seriously? You really need to know this in order to operate a computer? You should have seen the faces of my classmates. I consider myself lucky for being able to see through crap like that. Not everyone can.

  13. Mahesh April 12, 2018 at 11:11 pm #

    3 months back i started learning ML in bottom up approach. So many i felt bored and i stopped learning again i resumed. My study was intermittent. I felt Top down approach will work for me.
    Thank You
    Mahesh

    • Mahesh April 12, 2018 at 11:12 pm #

      *So many Times*

    • Jason Brownlee April 13, 2018 at 6:40 am #

      I’m glad to hear that Mahesh, stick with it!

  14. Germán Goñi Núñez April 21, 2018 at 8:37 am #

    Thank you for the amazing content that you create Jason.

  15. Sean O'Connor April 23, 2018 at 2:41 pm #

    There is some sort of discussion about similarity alignment and also the quirky topic of feedback alignment here:

    https://discourse.numenta.org/t/similarity-alignment-a-missing-link-between-structure-function-and-algorithms/3683

    I was also practicing html5 and I did a WHT toy example:

    http://md2020.eu5.org/wht1.html

    • Jason Brownlee April 23, 2018 at 2:55 pm #

      Great, how does this relate to the post exactly Sean?

  16. Vinícius Gonçalves de Oliveira April 26, 2018 at 7:41 am #

    Man, how can I post a gift for you from Brazil, due the insights contained in these texts?

    I’m very happy to know that many practioners including you share or have shared time ago the same difficulties that I’ve been crossing right now.

    Thanks for your motivational work and technical posts…

    You’re giving me a huge help.

    Top-Down forever….

    Thanks a lot again…

    Best regards

  17. Sharp Sight April 30, 2018 at 8:58 am #

    Strongly agree with this.

    I tell my students that it’s like being a race-car driver: you don’t need to be able to engineer the car to be able to drive it.

    For a practitioner, it’s almost always best to just learn to use ML tools in a practical way and then “backfill” the mathematics as you need to start fine-tuning models, or as you need to deepen your knowledge.

    I think the only major difference in our philosophies is that I strongly suggest that students master basic analysis first: http://www.sharpsightlabs.com/blog/machine-learning-prerequisite-isnt-math/

  18. Hongli May 2, 2018 at 7:59 am #

    I like you post very much!

    I think learning a programming language(any language), we always learn it in top-bottom fashion.
    Because in order to learn it from bottom-up, this would mean, we need to know how the hardware works, and the OS that operates on the hardware, then the language that written to run in the os, then the compiler, finally the language itself. This is not possible and not efficient for any human being. In order to learn, we don’t need to know how our brain instructs us on how to learn. It’s implicitly Top-down by design.

  19. Evi June 13, 2018 at 9:45 pm #

    Thank you for your post. I am currently getting a PhD in Informatics in Germany, where we do not have courses. My research area is currently time series analysis with deep learning.
    I actually come from physics, and I switched my area of research completely. In the first month of my PhD, I made myself read the free Deep Learning book by Bengio and Goodfellow. While I think it is a good book, I was infinitely bored reading it from morning until evening. And I lost a month, basically. Now I try to use a different approach. For example, I have recently hacked a variational convolutional autoencoder and made it work for my time series dataset. And now I am happy to read the original paper on VAEs. Similarly, to get a better understanding of my models, I am motivated to read books on statistics in my free time. But starting my PhD work with reading a book gave me nearly nothing. In fact, I probably should reread the Deep Learning book, since now, I would certainly get more out of it.

    • Jason Brownlee June 14, 2018 at 6:03 am #

      Well done.

      How was the skill of your VAE on your time series problem?

  20. Landon Eggleston June 20, 2018 at 2:24 am #

    I rarely leave replies to articles / blogposts, but the principal and analogies you provide here are spot on. I have a feeling that both your free and paid content will accelerate the learning curve of those that are already motivated to learn, and better yet – may provide the confidence to begin for potential ML learners who feel inadequate due to the perception of complexity, and the often assumed “requirement” of knowledge, both in depth and breadth even to start.

    — Start now, Fail often. Nothing motivates more than success learned from prior failure —

  21. Eric Pearce June 28, 2018 at 6:26 am #

    I’ve only been learning Machine Learning for about two weeks. I think I have been bouncing back and forth between the bottom up approach to trying to find examples that fit my problem (top down). I am sure the top down approach is the way to go, but my problem isn’t really a discrete problem, it is dealing with a very physical system, whose governing equations are simply to complex to distill into one neat little equation. As a result, I have what seems to me like a supervised system, and it is continuous. I have four inputs, two highly intertwined outputs and just can’t make sense of where to go now. Open to suggestions as I continue to look for examples and methods. Thanks for your site, just found it today and am binge reading… 😉

  22. Grant June 28, 2018 at 9:17 pm #

    This is a wonderful article, thank you for spelling it out like that.

    I have approached ML and pre reqs from the bottom-up so far, but realize that I could be much farther along if I had approached it differently. I have spent countless hours trying to work though compiler errors or equivalent roadblocks in other subjects and see that a lot of that was wasted time.

    This is my first year out of my near finished undergrad program; unfortunately it was not going to be able to deliver its bottom-up promise, and only delaying and indebting me.

    I’ve been trying to fill in my gaps with mostly bottom-up approach with studying on my own, but feel buried and lost a lot of the time. There are so many wonderful resources online, I’m trying to balance forward momentum and my tendency to want to know every intricate detail of something before moving on.

    Thank you for this great article and the site! I know this will be a treasured resource as I figure out the plan going forward. A top-down learning approach will surely be a better way for me!

    • Jason Brownlee June 29, 2018 at 5:59 am #

      I’m glad it helped Grant. Stick with it.

      I’m here to help if I can along your journey.

      Focus on working problems end to end and delivering results. It is the skeleton that will allow everything else to make sense, or fall away.

  23. Ahmed July 10, 2018 at 1:54 pm #

    if this your way of thinking about learn, then I’m happy that by luck I found you..

  24. Shailna Patidar October 15, 2018 at 5:30 pm #

    Hey Jason,

    I think now I can earn anything with your approach and not just programming. 🙂

  25. Ankur October 24, 2018 at 4:20 pm #

    Thanks for introducing the new approach. I was just thinking of myself of using top down approach and with this article I confirmed… Let’s see how it works…

  26. Ronnie October 31, 2018 at 1:14 am #

    I like the approach and I have to say everything you’ve mentioned make sense. Personally i hated college for their approach on education and always prefer my “way”, now i see my way as the Top-Down Learning method, and I learn many things that way without even know.

    I think it is time to tame Machine Learning then

  27. Tanuj Dave November 9, 2018 at 10:30 pm #

    It really awesome article to start on ML, Thanks Jason

Leave a Reply