You’re Doing it Wrong. Why Machine Learning Does Not Have to Be So Hard

Technical topics like mathematics, physics, and even computer science are taught using a bottom-up approach.

This approach involves laying out the topics in an area of study in a logical way with a natural progression in complexity and capability.

The problem is, humans are not robots executing a learning program. We require motivation, excitement, and most importantly, a connection of the topic to tangible results.

Useful skills we use every day like reading, driving, and programming were not learned this way and were in fact learned using an inverted top-down approach. This top-down approach can be used to learn technical subjects directly such as machine learning, which can make you a lot more productive a lot sooner, and be a lot of fun.

In this post, you will discover the concrete difference between the top-down and bottom-up approaches to learning technical material and why this is the approach that practitioners should use to learn machine learning and even related mathematics.

After reading this post, you will know:

  • The bottom-up approach used in universities to teach technical subjects and the problems with it.
  • How people learn to read, drive, and program in a top-down manner and how the top-down approach works.
  • The frame of machine learning and even mathematics using the top-down approach to learning and how to start to make rapid progress as a practitioner.

Let’s get started.

You're Doing it Wrong. Why Machine Learning Does Not Have to Be So Hard

You’re Doing it Wrong. Why Machine Learning Does Not Have to Be So Hard
Photo by popofatticus, some rights reserved.

Overview

This is an important blog post, because I think it can really help to shake you out of the bottom-up, university-style way of learning machine learning.

This post is divided into seven parts; they are:

  1. Bottom-Up Learning
  2. Learning to Read
  3. Learning to Drive
  4. Learning to Code
  5. Top-Down Learning
  6. Learn Machine Learning
  7. Learning Mathematics

Bottom-Up Learning

Take a field of study, such as mathematics.

There is a logical way to lay out the topics in mathematics that build on each other and lead through a natural progression in skills, capability, and understanding.

The problem is, this logical progression might only make sense to those who are already on the other side and can intuit the relationships between the topics.

Most of school is built around this bottom-up natural progression through material. A host of technical and scientific fields of study are taught this way.

Think back to high-school or undergraduate studies and the fundamental fields you may have worked through: examples such as:

  • Mathematics, as mentioned.
  • Biology.
  • Chemistry.
  • Physics.
  • Computer Science.

Think about how the material was laid out, week-by-week, semester-by-semester, year-by-year. Bottom-up, logical progression.

The problem is, the logical progression through the material may not be the best way to learn the material in order to be productive.

We are not robots executing a learning program. We are emotional humans that need motivation, interest, attention, encouragement, and results.

You can learn technical subjects from the bottom-up, and a small percentage of people do prefer things this way, but it is not the only way.

Now, if you have completed a technical subject, think back to how to you actually learned it. I bet it was not bottom-up.

Learning to Read

Think back; how did you learn to read?

My son is starting to read. Without thinking too much, here are the general techniques he’s using (really the school and us as parents):

  • Start by being read to in order to generate interest and show benefits.
  • Get the alphabet down and making the right sounds.
  • Memorize the most frequent words, their sounds, and how to spell them.
  • Learn the “spell-out-the-word” heuristic to deal with unknown words.
  • Read through books with supervision.
  • Read through books without supervision.

It is important that he continually knows why reading is important, connected to very tangible things he wants to do, like:

  • Read captions on TV shows.
  • Read stories on topics he loves, like Star Wars.
  • Read signs and menus when we are out and about.
  • So on…

It is also important that he gets results that he can track and in which he can see improvement.

  • Larger vocabulary.
  • Smoother reading style
  • Books of increasing complexity.

Here’s how he did not learn to read:

  • Definitions of word types (verbs, nouns, adverbs, etc.)
  • Rules of grammar.
  • Rules of punctuation.
  • Theory of human languages.

Learning to Drive

Do you drive?

It’s cool if you don’t, but most adults do out of necessity. Society and city design is built around personal mobility.

How did you learn to drive?

I remember some written tests and maybe a test on a computer. I have no memory of studying for them, though I very likely did. Here’s what I do remember.

I remember hiring a driving instructor and doing driving lessons. Every single lesson was practical, in the car, practicing the skill I was required to master, driving the vehicle in traffic.

Here’s what I did not study or discuss with my driving instructor:

  • The history of the automobile.
  • The theory of combustion engines.
  • The common mechanical faults in cars.
  • The electrical system of the car.
  • The theory of traffic flows.

To this day, I still manage to drive safely without any knowledge on these topics.

In fact, I never expect to learn these topics. I have zero need or interest and they will not help me realize the thing I want and need, which is safe and easy personal mobility.

If the car breaks, I’ll call an expert.

Learning to Code

I started programming without any idea of what coding or software engineering meant.

At home, I messed around with commands in Basic. I messed around with commands in Excel. I modified computer games. And so on. It was fun.

When I started to learn programming and software engineering, it was in university and it was bottom up.

We started with:

  • Language theory
  • Data types
  • Control flow structures
  • Data structures
  • etc.

When we did get to write code, it was on the command line and plagued with compiler problems, path problems, and a whole host of problems unrelated to actually learning programming.

I hated programming.

Flash-forward a few years. Somehow, I eventually starting working as a professional software engineer on some complex systems that were valued by their users. I was really good at it and I loved it.

Eventually, I did a course that showed how to create graphical user interfaces. And another that showed how to get computers to talk to each other using socket programming. And another on how to get multiple things to run at the same time using threads.

I connected the boring stuff with the thing I really liked: making software that could solve problems, that others could use. I connected it to something that mattered. It was no longer abstract and esoteric.

At least for me, and many developers like me, they taught it wrong. They really did. And it wasted years of time, effort, and results/outcomes that enthusiastic and time-free students like me could dedicate to something they are truly passionate about.

Top-Down Learning

The bottom-up approach is not just a common way for teaching technical topics; it looks like the only way.

At least until you think about how you actually learn.

The designers of university courses, masters of their subject area, are trying to help. They are laying everything out to give you the logical progression through the material that they think will get you to the skills and capabilities that you require (hopefully).

And as I mentioned, it can work for some people.

It does not work for me, and I expect it does not work for you. In fact, very few programmers I’ve met that are really good at their craft came through computer science programs, or if they did, they learned at home, alone, hacking on side projects.

An alternative is the top-down approach.

Flip the conventional approach on its head.

Don’t start with definitions and theory. Instead, start by connecting the subject with the results you want and show how to get results immediately.

Lay out a program that focuses on practicing this process of getting results, going deeper into some areas as needed, but always in the context of the result they require.

It Is Different

It is not the traditional path.

Be careful not to use traditional ways of thinking or comparison if you take this path.

The onus is on you. There is no system to blame. You only fail when you stop.

  • It is iterative. Topics are revisited many times with deeper understanding.
  • It is imperfect. Results may be poor in the beginning, but improve with practice.
  • It requires discovery. The learner must be open to continual learning and discoverery.
  • It requires ownership. The learner is responsible for improvement.
  • It requires curiosity. The learner must pay attention to what interests them and follow it.

It Is Dangerous

Seriously, I’ve heard “experts” say this many times, saying things like:

You have to know the theory first before you can use this technique, otherwise you cannot use it properly.

I agree that results will be imperfect in the beginning, but improvement and even expertise does not only have to come from theory and fundamentals.

If you believe that a beginner programmer should not be pushing changes to production and deploying them, then surely you must believe that a beginner machine learning practitioner would suffer the same constraints.

Skill must be demonstrated.

Trust must be earned.

This is true regardless of how a skill is acquired.

You’re a Technician

Really!?

This is another “criticism” I’ve seen leveled at this approach to learning.

Exactly. We want to be technicians, using the tools in practice to help people and not be researchers..

You do not need to cover all of the same ground because you have a different learning objective. Although you can circle back and learn anything you like later once you have a context in which to integrate the abstract knowledge.

Developers in industry are not computer scientists; they are engineers. They are proud technicians of the craft.

Efficient, Effective, and a Fun Way to Learn

The benefits vastly outweigh the challenge of learning this way:

  • You go straight to the thing you want and start practicing it.
  • You have a context for connecting deeper knowledge and even theory.
  • You can efficiently sift and filter topics based on your goals in the subject.

It’s faster.

It’s more fun.

And, I bet it makes you much better.

How could you be better?

Because the subject is connected to you emotionally. You have connected it to an outcome or result that matters to you. You are invested. You have demonstrable competence. We all love things we are good at (even if we are a little color blind to how good we are), which drives motivation, enthusiasm, and passion.

An enthusiastic learner will blow straight past the fundamentalist.

Learn Machine Learning

So, how have you approached the subject of machine learning?

Seriously, tell me your approach in the comments below.

  • Are you taking a bottom-up university course?
  • Are you modeling your learning on such a course?

Or worse:

Are you following a top-down type approach but are riddled with guilt, math envy, and insecurities?

You are not alone; I see this every single day in helping beginners on this website.

To connect the dots for you, I strongly encourage you to study machine learning using the top-down approach.

  • Don’t start with precursor math.
  • Don’t start with machine learning theory.
  • Don’t code every algorithm from scratch.

This can all come later to refine and deepen your understanding once you have connections for this abstract knowledge.

  1. Start by learning how to work through very simple predictive modeling problems using a fixed framework with free and easy-to-use open source tools.
  2. Practice on many small projects and slowly increase their complexity.
  3.  Show your work by building a public portfolio.

I have written about this approach many times; see the “Further Reading” section at the end of the post for some solid posts on how to get started with the top-down approach to machine learning.

“Experts” entrenched in universities will say it’s dangerous. Ignore them.

World-class practitioners will tell you it’s the way they learned and continue to learn. Model them.

Remember:

  • You learned to read by practicing reading, not by studying language theory.
  • You learned to drive by practicing driving, not by studying combustion engines.
  • You learned to code by practicing coding, not by studying computability theory.

You can learn machine learning by practicing predictive modeling, not by studying math and theory.

Not only is this the way I learned and continue to practice machine learning, but it has helped tens of thousands of my students (and the many millions of readers of this blog).

Learning Mathematics

Don’t stop there.

A time may come when you want or need to pull back the curtain on the mathematical pillars of machine learning such as linear algebra, calculus, statistics, probability, and so on.

You can use the exact same top-down approach.

Pick a goal or result that matters to you, and use that as a lens, filter, or sift on the topics to study and learn to the depth you need to get that result.

For example, let’s say you pick linear algebra.

A goal might be to grok SVD or PCA. These are methods used in machine learning for data projection, data reduction, and feature selection type tasks.

A top-down approach might be to:

  1. Implement the method in a high-level library such as scikit-learn and get a result.
  2. Implement the method in a lower-level library such as NumPy/SciPy and reproduce the result.
  3. Implement the method directly using matrices and matrix operations in NumPy or Octave.
  4. Study and explore the matrix arithmetic operations involved.
  5. Study and explore the matrix decomposition operations involved.
  6. Study methods for approximating the eigendecomposition of a matrix.
  7. And so on…

The goal provides the context and you can let your curiosity define the depth of study.

Painted this way, studying math is no different to studying any other topic in programming, machine learning, or other technical subjects.

It’s highly productive, and it’s a lot of fun!

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Summary

In this post, you discovered the concrete difference between the top-down and bottom-up approaches to learning technical material and why this is the approach that practitioners should and do use to learn machine learning and even related mathematics.

Specifically, you learned:

  • The bottom-up approach used in universities to teach technical subjects and the problems with it.
  • How people learn to read, drive, and program in a top-down manner and how the top-down approach works.
  • The frame of machine learning and even mathematics using the top-down approach to learning and how to start to make rapid progress as a practitioner.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

29 Responses to You’re Doing it Wrong. Why Machine Learning Does Not Have to Be So Hard

  1. The c0d3R March 28, 2018 at 6:05 am #

    Hi Jason,

    From my anecdotal experience I can say that the top down approach suited me more than the bottom up. I have taken a lot of ML & DL courses but I would always either loose interest or get so bogged down by the early complexity that it would deter me from actually learning the fun application’s part. I got way better with ML when I started applying models to real world problems.

    Thanks for confirming my bias!

  2. Will Bidstrup March 28, 2018 at 6:24 am #

    Thank you Jason,

    I believe this is excellent advice, at least for me. I find I need to oscillate between top down and bottoms up, but here’s the kicker – I can ONLY make myself learn bottoms up when I have motivation which I get from being enthused about doing something exciting like solving a real world problem or making something cool.

    I’m looking forward to working through some of your material and also beginning to populate my own learning blog.

    Thanks for the clear thinking and inspirational post.

    Will

  3. Lissy March 28, 2018 at 11:57 am #

    I agree, this is the best way to learn machine learning. I skipped all the math and dove into the fun stuff. Now I’m going back and taking a statistics course. I never would have mad it through if I started with statistics

  4. Jeff March 28, 2018 at 12:41 pm #

    I had the same experience with learning programming in college. Took a class in 2006, to learn C. Had to also use command line and Linux, which made it that much harder. Fast forward to 2012 – learned Java on iTunes U, by moving little robots around and making balls bounce… in an IDE, on my familiar computer. So much more fun. Went on to learn C#, Python, and R in the last few years by doing random little projects that helped me at work or automated something boring at home.

  5. Raj March 28, 2018 at 6:54 pm #

    Agree and thanks again for putting it in a simpler way.

  6. Muyi Ibidun March 28, 2018 at 10:06 pm #

    I totally agree with the top-down approach to learning especially in a fast pace field like computing science, data science and machine learning where techniques and technologies transforms faster than you could master the theories behind them. However, I have come to realise my background in computing has helped me in ways it has become really easy to grasp the underlying concepts behind the techniques on-demand.

  7. Dhrumil March 29, 2018 at 12:36 pm #

    I feel like I had an explosion.You articulated really well what I have been doing for a some time already but feeling fear,self-deceit,guilt,paranoia etc.I guess I won’t anymore.

    Thank you Jason.

  8. Tim March 29, 2018 at 3:38 pm #

    Thank you for writing this. I just discovered your blog and I will be reading everything you’ve written here. 2 months ago, I got super excited about Machine Learning and AI and did the usual googling rounds of “How do I get started in ML” and “Which languages should I learn for ML” and “Prerequisites for ML”.

    The results I got from that research shaped my decision of what to study and how to get started – I made a list of all the courses starting from the Ng ML course, got the Multivariable calculus, linear algebra, statistics books, and started teaching myself Python and Calculus. Two Months later? Still reading Python and Calculus books, solving problems, doing exercises (sigh.)

    I’m a little bit of a perfectionist (read: fan of bottom-up learning in theory). I mean, plan out everything, take every course, understand every underlying mathematical theory before attempting machine learning courses and EVERYTHING will go smoothly and I will master everything in the shortest amount of time, it’s totally supposed to work right?!

    Sometimes, I forget how I learned to read, make music, use a computer, or play a sport. Bottom-up is a beautiful idea, but it only works in a utopian world where I have unlimited motivation every day to master things that don’t directly interest me.

    I’m going to change my approach from today (read: throw everything out and play with stuff). I want the excitement back. Thanks again for writing this, Jason.

    • Jason Brownlee March 30, 2018 at 6:31 am #

      I’m glad the message helped Tim.

      Focus on providing value, being useful. You cannot learn everything, and if you go down that road, almost all of it will not help you to be useful to others (e.g. as a data scientist/ml engineer).

      Hang in there.

  9. Franco March 29, 2018 at 7:45 pm #

    The famous mathematician Carl Jacobi would probably agree with you. His maxim was “invert, always invert” (German: “man muss immer umkehren”) or WORK BACKWARD.

    Another deeply profound post Jason! Thank you.

    • Jason Brownlee March 30, 2018 at 6:36 am #

      Thanks Franco, I’m glad it helped.

      Thanks for sharing the quote!

  10. Patrick March 29, 2018 at 10:18 pm #

    Completely agree – great post!

  11. Doi April 6, 2018 at 8:24 pm #

    Thats such an aid.
    It seems I have been approaching things the wrong way.
    And then I read your article and it tells me why even after studying ml for so long around two years i havent been satisfied.
    I took a hybrid approach.

    I will get it sorted.
    Where should I get started?
    I cant start from very basic. I have iteratively left it incomplete.
    And cant approach it again.

    Suggestions please.

  12. BitBarrel April 11, 2018 at 7:36 pm #

    Very well said. It is a problem in many fields of education.

    I remember my first IT class in high school where the teacher explained how a floppy drive is divided into different sections. I mean, seriously? You really need to know this in order to operate a computer? You should have seen the faces of my classmates. I consider myself lucky for being able to see through crap like that. Not everyone can.

  13. Mahesh April 12, 2018 at 11:11 pm #

    3 months back i started learning ML in bottom up approach. So many i felt bored and i stopped learning again i resumed. My study was intermittent. I felt Top down approach will work for me.
    Thank You
    Mahesh

    • Mahesh April 12, 2018 at 11:12 pm #

      *So many Times*

    • Jason Brownlee April 13, 2018 at 6:40 am #

      I’m glad to hear that Mahesh, stick with it!

  14. Germán Goñi Núñez April 21, 2018 at 8:37 am #

    Thank you for the amazing content that you create Jason.

Leave a Reply