Introduction to Linear Algebra by Gilbert Strang for Machine Learning

If you are interested in getting started with or brushing up on the topic of linear algebra, then there is one textbook that is most likely going to be recommended to you.

That textbook is “Introduction to Linear Algebra” by Gilbert Strang and it provides a reference for his linear algebra course taught at MIT to undergraduate students.

In this post, you will discover the book “Introduction to Linear Algebra” by Gilbert Strang and how you can make the best use of it as a machine learning practitioner.

After reading this post, you will know:

  • About the goals and benefits of the book to a beginner or practitioner.
  • The contents of the book and general topics presented in each chapter.
  • A selected reading list targeted for machine learning practitioners looking to get up to speed fast.

Let’s get started.

Introduction to Linear Algebra by Gilbert Strang for Machine Learning

Introduction to Linear Algebra by Gilbert Strang for Machine Learning
Photo by Nayuki, some rights reserved.

Book Overview

Gilbert Strang teaches an introductory course to linear algebra at MIT.

His textbook titled “Introduction to Linear Algebra” is designed to support this course. His course and this textbook are widely regarded and often the first book recommended to undergraduate students looking to learn linear algebra.

Amazon Image

The book does assume some mathematical background, namely some calculus and familiarity with vectors and matrices.

18.02 Multivariable Calculus is a formal prerequisite for MIT students wishing to enroll in 18.06 Linear Algebra, but knowledge of calculus is not required to learn the subject. […] To succeed in this course you will need to be comfortable with vectors, matrices, and three-dimensional coordinate systems.

Prerequisites, Linear Algebra, MITOpenCourseware

His book is excellent, if not a little theoretical.

It can be used as a good starting point for machine learning practitioners interested in getting started or brushing up on their linear algebra. I think this is still the case if you do not have a background in calculus.

The century of data has begun! […] The truth is that vectors and matrices have become the language to know.

— Page ix, Introduction to Linear Algebra, Fifth Edition, 2016.

Concepts in the book are laid out clearly, often with diagrams, but the book moves quickly. The book expects you to keep up or you will fall behind.

That being said, each section has an overview of the concepts to be covered and ends with worked examples and quiz questions, the answers to which are available on the book’s website.

Need help with Linear Algebra for Machine Learning?

Take my free 7-day email crash course now (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

Download Your FREE Mini-Course

Table of Contents

This section provides a summary of the table of contents of the book.

  • Chapter 1: Introduction to Vectors
  • Chapter 2: Solving Linear Equations
  • Chapter 3: Vector Spaces and Subspaces
  • Chapter 4: Orthogonality
  • Chapter 5: Determinants
  • Chapter 6: Eigenvalues and Eigenvectors
  • Chapter 7: The Singular Value Decomposition (SVD)
  • Chapter 8: Linear Transformations
  • Chapter 9: Complex Vectors and Matrices
  • Chapter 10: Applications
  • Chapter 11: Numerical Linear Algebra
  • Chapter 12: Linear Algebra in Probability & Statistics

The book’s homepage provides a fuller summary including chapter sections.

The back cover provides a beautiful and elegant way of describing the goal of the book:

This book is designed to help students understand and solve the four central problems of linear algebra:

  • Ax = b n b n Chapters 1-2 Linear Systems
  • Ax = b m by n Chapters 3-4 Least Squares
  • Ax = lambda x n by b Chapters 5-6 Eigenvalues
  • Av = sigma u m by b Chapters 7-8 Singular values

Crash-Course For Machine Learning Practitioners
(a selected reading list)

The book is excellent and I recommend reading it from cover-to-cover, if you’re really into it.

But, as a machine learning practitioner, you do not need to read it all.

Below is a list of selected reading from the book that I recommend to get on top of linear algebra fast:

  • Section 1.1 Vectors and Linear Combinations
  • Section 1.2 Lengths and Dot Products
  • Section 1.3 Matrices
  • Section 2.4 Rules for Matrix Operations
  • Section 2.5 Inverse Matrices
  • Section 2.6 Elimination = Factorization: A = LU
  • Section 2.7 Transposes and Permutations
  • Section 4.3 Least Squares Approximations
  • Section 5.1 The Properties of Determinants
  • Section 6.1 Introduction to Eigenvalues
  • Section 6.2 Diagonalizing a Matrix
  • Section 6.4 Symmetric Matrices
  • Section 7.1 Image Processing by Linear Algebra
  • Section 7.2 Bases and Matrices in the SVD
  • Section 7.3 Principal Component Analysis (PCA by the SVD)
  • Section 12.1 Mean, Variance, and Probability
  • Section 12.2 Covariance Matrices and Joint Probabilities

Further, I would make the following recommendations:

  • Attempt the end of section questions and check your answers.
  • Consider implementing the methods directly in Python using NumPy function calls.
  • Research and find examples where some or all of these operations are used in machine learning algorithms, papers, or textbooks.

Did you explore these extensions?
Post your findings in the comments below.

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Summary

In this post, you discovered the book “Introduction to Linear Algebra” by Gilbert Strang and how you can make the best use of it as a machine learning practitioner.

Specifically, you learned:

  • About the goals and benefits of the book to a beginner or practitioner.
  • The contents of the book and general topics presented in each chapter.
  • A selected reading list targeted for machine learning practitioners looking to get up to speed fast.

Have you read this book? What did you think?
Let me know in the comments below.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.


Get a Handle on Linear Algebra for Machine Learning!

Linear Algebra for Machine Learning

Develop a working understand of linear algebra

…by writing lines of code in python

Discover how in my new Ebook:
Linear Algebra for Machine Learning

It provides self-study tutorials on topics like:
Vector Norms, Matrix Multiplication, Tensors, Eigendecomposition, SVD, PCA and much more…

Finally Understand the Mathematics of Data

Skip the Academics. Just Results.

Click to learn more.


6 Responses to Introduction to Linear Algebra by Gilbert Strang for Machine Learning

  1. Ataliba Miguel March 19, 2018 at 11:10 pm #

    Hi Jason,
    It’s very interesting all the brush up linear algebra towards machine learning coding. I am wondering if is there something similar but instead of Python using Matlab.
    I came across Machine Learning course provided by Stanford Uni via Coursera by Andrew, but I wanted to improve further.
    Regards

  2. Sean O'Connor March 22, 2018 at 2:00 pm #

    I read that book a very long time ago. It’s the greatest book ever on linear algebra. It’s so useful, practical and interesting.

  3. Craig Holley March 23, 2018 at 7:13 am #

    Just ordered the book based on your recommendation (and those on Amazon). Need to refresh all this it’s been too long. Jason I am using it in combination with your great book on Linear Algebra the pythonic way…..really an excellent document in it’s own right.

    • Jason Brownlee March 23, 2018 at 8:28 am #

      Thanks Craig!

      The Strang book is an excellent reference.

Leave a Reply