Basics of Linear Algebra for Machine Learning
Discover the Mathematical Language of Data in Python
Linear algebra is a pillar of machine learning. You cannot develop a deep understanding and application of machine learning without it.
In this new laser-focused Ebook written in the friendly Machine Learning Mastery style that you’re used to, you will finally cut through the equations, Greek letters, and confusion, and discover the topics in linear algebra that you need to know.
Using clear explanations, standard Python libraries, and step-by-step tutorial lessons, you will discover what linear algebra is, the importance of linear algebra to machine learning, vector, and matrix operations, matrix factorization, principal component analysis, and much more.
About the Ebook:
- PDF format Ebook.
- 6 parts, covering the main topics.
- 19 step-by-step tutorials.
- 211 pages.
- 92 Python (.py) code files included.
Clear and Complete Examples.
Designed for Developers. Nothing Hidden.
Click to jump straight to the packages.
Why Linear Algebra?
Linear algebra is a sub-field of mathematics concerned with vectors, matrices, and operations on these data structures. It is absolutely key to machine learning.
As a machine learning practitioner, you must have an understanding of linear algebra.
It’s a huge field of study that has made an impact on other fields, such as statistics, as well as engineering and physics. Thankfully, we don’t need to know the breadth and depth of the field of linear algebra in order to improve our understanding and application of machine learning.
Frustrated with the Math?
Have you ever been frustrated reading the description of a machine learning technique?
You’re reading along, things are going well, and then you hit an equation, and you are stopped in your tracks with questions like:
- … what do the terms mean?
- … why are there no operators between terms?
- … what does this Greek letter mean?
Unless you have a basic knowledge of linear algebra, you will not be able to read and understand even the most basic equations.
Have you heard of TensorFlow, Google’s Python library for deep learning?
Did you know that “tensor” is a term taken directly from the field of linear algebra and it simply means an array with more than two-dimensions?
Why Is Linear Algebra Important to Machine Learning?
So, why is linear algebra used so much to describe machine learning algorithms?
Linear algebra is about vectors and matrices and in machine learning we are always working with vectors and matrices (arrays) of data.
Linear algebra is essentially the mathematics of data.
It provides useful shortcuts for describing data as well as operations on data that we need to perform in machine learning methods.
Linear algebra is not magic
And, linear algebra is not trying to be exclusive or opaque.
As a first step, think of linear algebra as a shortcut language or notation to make describing some operations compact.
There are common machine learning techniques that can only be understood via their linear algebra descriptions because they come from the field. Techniques such as:
- Singular-Value Decomposition, or SVD.
- Principal Component Analysis.
- Linear Least Squares for Linear Regression.
Without a basic understanding of matrices and matrix operations, an understanding of these techniques will elude you.
The 3 Mistakes Made By Beginners
Once you discover the importance of linear algebra to machine learning, there are three key mistakes that beginners make:
1. Beginners Study Linear Algebra Too Early
If you ask how to get started in machine learning, you will very likely be told to start with linear algebra.
We know that knowledge of linear algebra is critically important, but it does not have to be the place to start. Learning linear algebra first, then calculus, probability, statistics, and eventually machine learning theory is a long and slow bottom-up path.
A better fit for developers is to start with systematic procedures that get results, and work back to the deeper understanding of theory, using working results as a context. I call this the top-down, or results-first, approach to machine learning, and linear algebra is not the first step, but perhaps the second or third.
2. Beginners Study Too Much Linear Algebra
When practitioners do circle back to study linear algebra, they learn far more of the field than is required for or relevant to machine learning.
Linear algebra is a large field of study that has tendrils into engineering, physics, and quantum physics. There are also theorems and derivations for nearly everything, most of which will not help you get better skill from, or a deeper understanding of, your machine learning model.
Only a specific subset of linear algebra is required, though you can always go deeper once you have the basics.
3. Beginners Study Linear Algebra Wrong
Linear algebra textbooks will teach you linear algebra in the classical university bottom-up approach. This is too slow (and painful) for your needs as a machine learning practitioner.
Like learning machine learning itself, take the top-down approach. Rather than starting with theorems and abstract concepts, you can learn the basics of linear algebra in a concrete way with data structures and worked examples of operations on those data structures. It’s so much faster (and fun).
Once you know how operations work, you can circle back and learn how they were derived. If you’re interested.
A Better Way into Linear Algebra
I am frustrated at seeing practitioner after practitioner diving into linear algebra textbooks and online courses designed for undergraduate students and giving up. The bottom-up approach is hard, especially if you already have a full time job.
Linear algebra is not only important to machine learning, but it is also a lot of fun, or can be if it is approached in the right way.
I want to help you see the field the way I see it: as just another set of tools we can harness on our journey toward machine learning mastery.
5 Areas of Linear Algebra to Focus On
You don’t need to know all of linear algebra.
The five key areas of linear algebra that I recommend you focus on are:
1. Learn Linear Algebra Notation
You need to be able to read and write vector and matrix notation.
Algorithms are described in books, papers, and on websites using vector and matrix notation.
Linear algebra is the mathematics of data and the notation allows you to describe operations on data precisely with specific operators.
You need to be able to read and write this notation.
2. Learn Linear Algebra Arithmetic
In partnership with the notation of linear algebra are the arithmetic operations performed.
You need to know how to add, subtract, and multiply scalars, vectors, and matrices.
A challenge for newcomers to the field of linear algebra are operations such as matrix multiplication and tensor multiplication that are not implemented as the direct multiplication of the elements of these structures, and at first glance appear nonintuitive.
3. Learn Linear Algebra for Statistics
You must learn linear algebra in order to be able to learn statistics. Especially multivariate statistics.
Statistics is concerned with describing and understanding data. As the mathematics of data, linear algebra has left its fingerprint on many related fields of mathematics, including statistics.
In order to be able to read and interpret statistics, you must learn the notation and operations of linear algebra.
4. Learn Matrix Factorization
Building on notation and arithmetic is the idea of matrix factorization, also called matrix decomposition.
You need to know how to factorize a matrix and what it means.
Matrix factorization is a key tool in linear algebra and used widely as an element of many more complex operations in both linear algebra (such as the matrix inverse) and machine learning (least squares, PCA, SVD, and more).
5. Learn Linear Least Squares
You need to know how to use matrix factorization to solve linear least squares.
Linear algebra was originally developed to solve systems of linear equations. These are equations where there are more equations than there are unknown variables. As a result, they are challenging to solve arithmetically because there is no single solution as there is no line or plane that can fit the data without some error.
Problems of this type can be framed as the minimization of squared error, called least squares, and can be recast in the language of linear algebra, called linear least squares.
If I could give one more reason, it would be: because it is fun.
Learning linear algebra, at least the way I teach it with practical examples and executable code, is a lot of fun. Once you can see how the operations work on real data, it is hard to avoid developing a strong intuition for the methods.
Introducing My New Ebook:
“Basics of Linear Algebra for Machine Learning“
Welcome to the “Basics of Linear Algebra for Machine Learning”
I designed this book to teach machine learning practitioners, like you, step-by-step the basics of linear algebra with concrete and executable examples in Python.
Who Is This Book For?
…so is this book right for YOU?
This book is for developers that may know some applied machine learning.
Maybe you know how to work through a predictive modeling problem end-to-end, or at least most of the main steps, with popular tools.
The lessons in this book do assume a few things about you, such as:
- You may know your way around basic Python for programming.
- You may know some basic NumPy for array manipulation.
- You want to learn linear algebra to deepen your understanding and application of machine learning.
This guide was written in the top-down and results-first machine learning style that you’re used to from Machine Learning Mastery.
What if I Am New to Machine Learning?
This book does not assume you have a background in machine learning.
That being said, I do recommend that you learn how to work through a predictive modeling problem first. It will give you the context for linear algebra. Otherwise the topic will feel too abstract.
What if I Am Just a Developer?
Perfect. I wrote this book for you.
What if My Math is Really Poor?
Maybe you learned linear algebra a long time ago back in school?
Maybe you never covered linear algebra before.
Perfect. This book is for you. I assume you know some basic arithmetic, and even then I give you a refresher.
What if I Am Not a Python Programmer?
You can handle this book if you are a programmer in another language, even if you are not experienced in Python.
Everything is demonstrated with a small code example that you can run directly.
All code is provided for you to play with, modify, and learn from.
I even show you how to manipulate NumPy arrays from first principles, because that is how we do linear algebra in Python.
The book even has an appendix to show you how to set up Python on your workstation.
What if I Am Working Through a Linear Algebra Course at a University?
This book is not a substitute for an undergraduate course in linear algebra or a textbook for such a course, although it is a great complement to such materials.
About Your Outcomes
…so what will YOU know after reading this book?
After reading and working through this book, you will know:
- What linear algebra is and why it is relevant and important to machine learning.
- How to create, index, and generally manipulate data in NumPy arrays.
- What a vector is and how to perform vector arithmetic and calculate vector norms.
- What a matrix is and how to perform matrix arithmetic, including matrix multiplication.
- A suite of types of matrices, their properties, and advanced operations involving matrices.
- What a tensor is and how to perform basic tensor arithmetic.
- Matrix factorization methods, including the eigendecomposition and singular-value decomposition.
- How to calculate and interpret basic statistics using the tools of linear algebra.
- How to implement methods using the tools of linear algebra such as principal component analysis and linear least squares regression.
This new basic understanding of linear algebra will impact your practice of machine learning.
After reading this book, you will be able to:
- Read the linear algebra mathematics in machine learning papers.
- Implement the linear algebra descriptions of machine learning algorithms.
- Describe your machine learning models using the notation and operations of linear algebra.
What Exactly Is in This Book?
This book was designed to be a crash course in linear algebra for machine learning practitioners. Ideally, those with a background as a developer.
This book was designed around major data structures, operations, and techniques in linear algebra that are directly relevant to machine learning algorithms.
There are a lot of things you could learn about linear algebra, from theory to abstract concepts to APIs. My goal is to take you straight to developing an intuition for the elements you must understand with laser-focused tutorials.
I designed the tutorials to focus on how to get things done with linear algebra. They give you the tools to both rapidly understand and apply each technique or operation.
Each tutorial is designed to take you about one hour to read through and complete, excluding the extensions and further reading.
You can choose to work through the lessons one per day, one per week, or at your own pace. I think momentum is critically important, and this book is intended to be read and used, not to sit idle.
I would recommend picking a schedule and sticking to it.
The tutorials are divided into five parts:
- Part 1: Foundation. Discover a gentle introduction to the field of linear algebra and the relationship it has with the field of machine learning.
- Part 2: NumPy. Discover NumPy tutorials that show you how to create, index, slice, and reshape NumPy arrays, the main data structure used in machine learning and the basis for linear algebra examples in this book.
- Part 3: Matrices. Discover the key structures for holding and manipulating data in linear algebra in vectors, matrices, and tensors.
- Part 4: Factorization. Discover a suite of methods for decomposing a matrix into its constituent elements in order to make numerical operations more efficient and more stable.
- Part 5: Statistics. Discover statistics through the lens of linear algebra and its application to principal component analysis and linear regression.
Below is an overview of the 19 step-by-step tutorial lessons you will work through:
Each lesson was designed to be completed in about 30-to-60 minutes by the average developer.
- Lesson 01: Introduction to Linear Algebra
- Lesson 02: Linear Algebra and Machine Learning
- Lesson 03: Examples of Linear Algebra in Machine Learning
- Lesson 04: Introduction to NumPy Arrays
- Lesson 05: Index, Slice, and Reshape NumPy Arrays
- Lesson 06: NumPy Array Broadcasting
- Lesson 07: Vectors and Vector Arithmetic
- Lesson 08: Vector Norms
- Lesson 09: Matrices and Matrix Arithmetic
- Lesson 10: Types of Matrices
- Lesson 11: Matrix Operations
- Lesson 12: Sparse Matrices
- Lesson 13: Tensors and Tensor Arithmetic
- Lesson 14: Matrix Decompositions
- Lesson 15: Eigendecomposition
- Lesson 16: Singular Value Decomposition
- Lesson 17: Introduction to Multivariate Statistics
- Lesson 18: Principal Component Analysis
- Lesson 19: Linear Regression
- Appendix A: Getting Help
- Appendix B: How to Set up a Workstation for Python
- Appendix C: Linear Algebra Cheat Sheet
- Appendix D: Basic Math Notation
You can see that each part targets a specific learning outcome, and so does each tutorial within each part. This acts as a filter to ensure you are only focused on the things you need to know to get to a specific result and do not get bogged down in the math or near-infinite number of digressions.
The tutorials were not designed to teach you everything there is to know about each of the theories or techniques of linear algebra. They were designed to give you an understanding of how they work, how to use them, and how to interpret the results the fastest way I know how: to learn by doing.
Table of Contents
The screenshot below was taken from the PDF Ebook. It provides you a full overview of the table of contents from the book.
Take a Sneak Peek Inside The Ebook
Click image to Enlarge.
Download Your Sample Chapter
Do you want to take a closer look at the book? Download a free sample chapter PDF.
Enter your email address and your sample chapter will be sent to your inbox.
BONUS: Linear Algebra Python Code Recipes
…you also get 92 fully working Python scripts
Sample Code Recipes
Each recipe presented in the book is standalone, meaning that you can copy and paste it into your project and use it immediately.
- You get one Python script (.py) for each example provided in the book.
This means that you can follow along and compare your answers to a known working implementation of each example in the provided Python files.
This helps a lot to speed up your progress when working through the details of a specific task, such as:
- Creating and manipulating NumPy arrays.
- Arithmetic with vectors and matrices.
- Advanced matrix operations.
- Matrix factorization methods
- Statistical methods with matrices.
The provided code was developed in a text editor and is intended to be run on the command line. No special IDE or notebooks are required.
All code examples were designed and tested with Python 3.6+.
All code examples will run on modest and modern computer hardware and were executed on a CPU.
Python Technical Details
This section provides some technical details about the code provided with the book.
- Python Version: You can use Python 3.6 or higher.
- SciPy: You will use NumPy and scikit-learn.
- Operating System: You can use Windows, Linux, or Mac OS X.
- Hardware: A standard modern workstation will do.
- Editor: You can use a text editor and run the example from the command line.
Don’t have a Python environment?
The appendix contains step-by-step tutorials showing you exactly how to set up a Python machine learning environment.
Check Out What Customers Are Saying:
Discover the Mathematics of Data TODAY!
Choose Your Package:
You will get the Ebook:
- Linear Algebra for Machine Learning
(including bonus source code)
(a great deal!)
All prices are in US Dollars (USD).
(1) Click the button. (2) Enter your details. (3) Download immediately.
Secure Payment Processing With SSL Encryption
Are you a Student, Teacher or Retiree?
Do you have any Questions?
About The Author
Hi, I'm Jason Brownlee.
I live in Australia with my wife and son and love to write and code.
I have a computer science background as well as a Masters and Ph.D. degree in Artificial Intelligence.
I’ve written books on algorithms, won and ranked in the top 10% in machine learning competitions, consulted for startups and spent a long time working on systems for forecasting tropical cyclones. (yes I have written tons of code that runs operationally)
I get a lot of satisfaction helping developers get started and get really good at machine learning.
I teach an unconventional top-down and results-first approach to machine learning where we start by working through tutorials and problems, then later wade into theory as we need it.
I'm here to help if you ever have any questions. I want you to be awesome at machine learning.
What Are Skills in Machine Learning Worth?
Your boss asks you:
Hey, can you build a predictive model for this?
Imagine you had the skills and confidence to say:
...and follow through.
I have been there. It feels great!
How much is that worth to you?
The industry is demanding skills in machine learning.
The market wants people that can deliver results, not write academic papers.
Business knows what these skills are worth and are paying sky-high starting salaries.
A Data Scientists Salary Begins at:
$100,000 to $150,000.
A Machine Learning Engineers Salary is Even Higher.
What Are Your Alternatives?
You made it this far.
You're ready to take action.
But, what are your alternatives? What options are there?
(1) A Theoretical Textbook for $100+
...it's boring, math-heavy and you'll probably never finish it.
(2) An On-site Boot Camp for $10,000+
...it's full of young kids, you must travel and it can take months.
(3) A Higher Degree for $100,000+
...it's expensive, takes years, and you'll be an academic.
For the Hands-On Skills You Get...
And the Speed of Results You See...
And the Low Price You Pay...
Machine Learning Mastery Ebooks are
And they work. That's why I offer the money-back guarantee.
You're A Professional
The field moves quickly,
...how long can you wait?
You think you have all the time in the world, but...
- New methods are devised and algorithms change.
- New books get released and prices increase.
- New graduates come along and jobs get filled.
Right Now is the Best Time to make your start.
Bottom-up is Slow and Frustrating,
...don't you want a faster way?
Can you really go on another day, week or month...
- Scraping ideas and code from incomplete posts.
- Skimming theory and insight from short videos.
- Parsing Greek letters from academic textbooks.
Targeted Training is your Shortest Path to a result.
Professionals Use Training To Stay On Top Of Their Field
Get The Training You Need!
You don't want to fall behind or miss the opportunity.
Frequently Asked Questions
Why doesn't my payment work?
I am sorry to hear that you're having difficulty.
- Perhaps you can double check that your details are correct, just in case of a typo?
- Perhaps you could try a different payment method, such as PayPal or Credit Card?
- Perhaps you could try my alternative secure payment processor, click here?
- Perhaps you're able to talk to your bank, just in case they blocked the transaction?
If you're still having difficulty, please contact me and I can help investigate further.
Can I get your books for free?
Sorry, I don’t give away free copies of my books.
You can access all of my best free material on my blog.
Can I get a hard copy of your book?
Sorry, I don't have hard copies by design.
The books are written for immediate use, rather than references to sit on the shelf.
My students like to have the PDF open on their screen next to their editor so they can copy-paste code.
Also, the books are updated often to reflect changes to APIs. The field is moving very fast.
I hope that helps explain the rationale.
Are there Kindle or ePub versions of the books?
Sorry, just PDF Ebooks.
This is by design and I put a lot of thought into it. My rationale is as follows:
- I use LaTeX to layout the text and code to give a professional look and I am afraid that EBook readers would mess this up.
- The increase in supported formats would create a maintenance headache that would take a large amount of time away from updating the books and working on new books.
- Most critically, reading on an e-reader or iPad is antithetical to the book-open-next-to-code-editor approach the PDF format was chosen to support.
My materials are playbooks intended to be open on the computer, next to a text editor and a command line. They are not reference texts to be read away from the computer.
Will I get free updates to the books?
All updates are free.
Books are usually updated once every month or two to fix bugs, typos and keep abreast of API changes.
Contact me anytime and check if there have been updates. Let me know what version of the book you have (version is listed on the copyright page).
How do I get access to any bonuses?
After you complete your purchase you will receive an email with a link to download your bundle.
The download will include the book or books and any bonus material.
Is there any digital rights management (DRM)?
Can I print the PDF for my personal use?
In what order should I read your books?
My best advice is to pick a topic that most interests you and start there.
Can I get a customized bundle of books?
Sorry, I cannot create custom bundles of books for you, it would create a maintenance nightmare for me. I’m sure you can understand.
You can see the full catalog of my books and bundles here.
Can I get an evaluation copy of your books?
Sorry, I no longer distribute evaluation copies of my books due to some past abuse of the privilege.
If you are a teacher or lecturer, I’m happy to offer you a student discount.
Contact me and ask for the discount.
Can I get an invoice for my purchase?
Email me with the details of your order (order number or email address used to make the purchase) and details you would like to appear on the invoice (your name, company name and address).
I will create a PDF invoice for you and email it back.
How long do books take to ship?
There are no physical books, therefore no shipping is required.
All books are EBooks that you can download immediately after you complete your purchase.
Do you ship to my country?
There are no physical books, therefore no shipping is required.
All books are EBooks that you can download immediately after you complete your purchase.
I support purchases from any country via PayPal or Credit Card.
Can I have a discount?
I do offer a discount to students, teachers, and retirees.
Note: I only offer discounts on individual books, not on the bundles. This is because the bundles are already heavily discounted.
If you are a student, teacher or a retiree please contact me and ask for the discount.
Do you have any sales, deals, or coupons?
I generally don't do sales.
If I do have a special, such as around the launch of a new book, I only offer it to past customers and subscribers on my email list.
I do offer book bundles that offer a discount for a collection of related books.
Can I get a refund?
I am sorry to hear that you want a refund.
Please contact me directly with your purchase details (order number or email address used to make the purchase) and I will organize a refund.
Will you help me if I have questions?
Please contact me anytime with questions about machine learning or the books.
One question at a time please.
Also, each book has a final chapter on getting more help and further reading and points to resources that you can use to get more help.
Do I need to be a good programmer?
Not at all.
My material requires that you have a programmers mindset of thinking in procedures and learning by doing.
You do not need to be an excellent programmer to read and learn about machine learning algorithms.
How much math do I need to know?
No background in statistics, probability or linear algebra is required.
I teach using a top-down and results-first approach to machine learning. You will learn by doing, not learn by theory.
There are no derivations.
Any questions presented are explained in full and are only provided to make the explanation clearer, not more confusing.
How much machine learning do I need to know?
Only a little.
If you are a reader of my blog posts, then you know enough to get started.
I do my best to lead you through what you need to know, step-by-step.
How long will the book take me to complete?
I recommend reading one chapter per day.
Some students finish the book in a weekend.
Most students finish the book in a few weeks by working through it during nights and weekends.
How are your books different to other books?
My books are playbooks. Not textbooks.
They have no deep explanations of theory, just working examples that are laser-focused on the information that you need to know to bring machine learning to your project.
My books are not for everyone, they are carefully designed for practitioners that need to get results, fast.
How are your books different from the blog?
The books are a concentrated and more convenient version of what I put on the blog.
I design my books to be a combination of lessons and projects to teach you how to use a specific machine learning tool or library and then apply it to real predictive modeling problems.
The books get updated with bug fixes, updates for API changes and the addition of new chapters, and these updates are totally free.
I do put some of the book chapters on the blog as examples, but they are not tied to the surrounding chapters or the narrative that a book offers and do not offer the standalone code files.
With each book, you also get all of the source code files used in the book that you can use as recipes to jump-start your own predictive modeling problems.
How are the 2 algorithms books different?
The book “Master Machine Learning Algorithms” is for programmers and non-programmers alike that learn through worked examples. It teaches you how 10 top machine learning algorithms work, with worked examples in arithmetic, not code (and spreadsheets) that show how each model learns and makes predictions.
The book “Machine Learning Algorithms From Scratch” is for programmers that learn by writing code to understand. It provides step-by-step tutorials on how to implement top algorithms as well as how to load data, evaluate models and more. It has less on how the algorithms work, instead focusing exclusively on how to implement each in code.
The two books can support each other.
Is there a team or company-wide license?
Due to abuse of the privilege, I only support purchases by individuals.
Is there a license for libraries?
Sorry, I only support purchases by individuals.
Do you have videos?
I only have tutorial lessons and projects in text format.
This is by design. I used to have video content and I found the completion rate much lower.
I want you to put the material into practice. I have found that text-based tutorials are the best way of achieving this.
After reading and working through the tutorials you are far more likely to apply what you have learned.
What operating systems are supported?
Linux, Mac OS X and Windows.
Can you be my mentor or coach?
Thanks for asking. I would love to help, but I just don't have the capacity.
I try to help as many people as possible through my blog and books.
Can I purchase from Amazon (or elsewhere)?
My books can only be purchased from my website.
The reason is that I am a small business and I want a direct relationship with you, my customer, so that I can offer personal support and send out updates about your book and new stuff I am working on.
I hope you can understand my rationale.
What if my download link expires?
It is possible that your link to download your purchase will expire after a few days.
This is a security precaution.
Please contact me and I will resend you purchase receipt with an updated download link.
Can I use your code in my own project?
But, understand that all code was developed and provided for educational purposes only and that I take no responsibility for it, what it might do or how you might use it.