SALE! Use code blackfriday for 40% off everything!
Hurry, sale ends soon! Click to see the full catalog.

Understanding Traceback in Python

Last Updated on June 21, 2022

When an exception occurs in a Python program, often a traceback will be printed. Knowing how to read the traceback can help you easily identify the error and make a fix. In this tutorial, we are going to see what the traceback can tell you.

After completing this tutorial, you will know:

  • How to read a traceback
  • How to print the call stack without an exception
  • What is not shown in the traceback

Kick-start your project with my new book Python for Machine Learning, including step-by-step tutorials and the Python source code files for all examples.

Let’s get started.

Understanding Traceback in Python
Photo by Marten Bjork, some rights reserved

Tutorial Overview

This tutorial is divided into four parts; they are:

  1. The call hierarchy of a simple program
  2. Traceback upon exception
  3. Triggering traceback manually
  4. An example in model training

The call hierarchy of a simple program

Let’s consider a simple program:

This program will print the Python dictionary data with indentations. Its output is the following:

This is a short program, but functions are calling each other. If we add a line at the beginning of each function, we can reveal how the output is produced with the flow of control:

and the output will be messed with more information:

So now we know the order of how each function is invoked. This is the idea of a call stack. At any point in time, when we run a line of code in a function, we want to know what invoked this function.

Traceback upon exception

If we make one typo in the code like the following:

The typo is at the last line, where the closing bracket should be at the end of the line, not before any +. The return value of the print() function is a Python None object. And adding something to None will trigger an exception.

If you run this program using the Python interpreter, you will see this:

The lines starting with “Traceback (most recent call last):” are the traceback. It is the stack of your program at the time when your program encountered the exception. In the above example, the traceback is in the “most recent call last” order. Hence your main function is at the top while the one triggering the exception is at the bottom. So we know the issue is inside the function printdict().

Usually, you will see the error message at the end of the traceback. In this example, it is a TypeError triggered by adding None and string. But the traceback’s help stops here. You need to figure out which one is None and which one is string. By reading the traceback, we also know the exception-triggering function printdict() is invoked by indentprint(), and it is in turn invoked by printlist(), and so on.

If you run this in a Jupyter notebook, the following is the output:

The information is essentially the same, but it gives you the lines before and after each function call.

Want to Get Started With Python for Machine Learning?

Take my free 7-day email crash course now (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

Triggering traceback manually

The easiest way to print a traceback is to add a raise statement to manually create an exception. But this will also terminate your program. If we want to print the stack at any time, even without any exception, we can do so with the following:

The line traceback.print_stack() will print the current call stack.

But indeed, we often want to print the stack only when there is an error (so we learn more about why it is so). The more common use case is the following:

This is a typical pattern for repeatedly calculating a function, such as Monte Carlo simulation. But if we are not careful enough, we may run into some error, such as in the above example where we may have division by zero. The problem is, in the case of more complicated computations, you can’t easily spot the flaw. Such as in the above, the issue is buried inside the call to compute(). Therefore, it is helpful to understand how we get the error. But at the same time, we want to handle the case of the error rather than let the entire program terminate. If we use the try-catch construct, the traceback will not be printed by default. Therefore, we need to use the traceback.print_exc() statement to do it manually.

Actually, we can have the traceback more elaborated. Because the traceback is the call stack, we can examine each function in the call stack and check the variables in each level. In this complicated case, this is the function I usually use to do a more detailed trace:

An example of model training

The call stack, as reported in the traceback, has a limitation: You can only see the Python functions. It should be just fine for the program you wrote, but many large libraries in Python have part of them written in another language and compiled into binary. An example is Tensorflow. All the underlying operations are in binary for the performance. Hence if you run the following code, you will see something different:

The input_shape parameter to the first LSTM layer in the model should be (n_in, 1) to match the input data, rather than (n_in+1, 1). This code will print the following error once you invoke the last line:

If you look at the traceback, you cannot really see the complete call stack. For example, the top frame you know you called model.fit(), but the second frame is from a function named error_handler(). Here, you cannot see how the fit() function triggered that. This is because Tensorflow is highly optimized. A lot of stuff is hidden in compiled code and not visible by the Python interpreter.

In this case, it is essential to patiently read the traceback and find the clue to the cause. Of course, the error message usually should also give you some useful hints.

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Books

Python Official Documentation

Summary

In this tutorial, you discovered how to read and print the traceback from a Python program.

Specifically, you learned:

  • What information the traceback tells you
  • How to print a traceback at any point of your program without raising an exception

In the next post, we will see how to navigate the call stack inside the Python debugger.

Get a Handle on Python for Machine Learning!

Python For Machine Learning

Be More Confident to Code in Python

...from learning the practical Python tricks

Discover how in my new Ebook:
Python for Machine Learning

It provides self-study tutorials with hundreds of working code to equip you with skills including:
debugging, profiling, duck typing, decorators, deployment, and much more...

Showing You the Python Toolbox at a High Level for
Your Projects


See What's Inside

2 Responses to Understanding Traceback in Python

  1. Manza Zityab December 31, 2021 at 4:09 pm #

    Thank you very much .I try to practice on this bash script . When I got some problem I will ask you dear .

    • James Carmichael January 1, 2022 at 12:11 pm #

      You are very welcome, Manza! Keep up the great work!

Leave a Reply