Understanding Traceback in Python

Last Updated on December 24, 2021

When an exception occurs in a Python program, often a traceback will be printed. Knowing how to read the traceback can help you easily identify the error and make a fix. In this tutorial we are going see what the traceback can tell you.

After completing this tutorial, you will know:

  • How to read a traceback
  • How to print the call stack without exception
  • What is not shown in the traceback

Let’s get started.

Understanding Traceback in Python
Photo by Marten Bjork, some rights reserved

Tutorial Overview

This tutorial is divided into 4 parts; they are:

  1. The call hierarchy of a simple program
  2. Traceback upon exception
  3. Triggering traceback manually
  4. An example in model training

The call hierarchy of a simple program

Let’s consider a simple program:

This program is to print the Python dictionary data with indentations. It’s output is the following:

This is a short program but functions are calling each other. If we add a line at the beginning of each function, we can reveal how the output is produced with the flow of control:

and the output will be messed with more information:

So now we knows the order of how each function is invoked. This is the idea of a call stack. At any point of time, when we run a line of code in a function, we want to know who invoked this function.

Traceback upon exception

If we make one typo in the code like the following:

The typo is at the last line, which the closing bracket should be at the end of line, not before any +. The return value of print() function is a Python None object. And adding something to None will trigger an exception.

If you run this program using Python interpreter, you will see this:

The lines starting with “Traceback (most recent call last):” is the traceback. It is the stack of your program at the time when your program encountered the exception. In the above example, the traceback is in “most recent call last” order. Hence your main function is at top while the one triggering the exception is at bottom. So we know the issue is inside the function printdict().

Usually you will see the error message at the end of the traceback. In this example, it is a TypeError triggered by adding None and string. But the traceback’s help stops here. You need to figure out which one is None and which one is string. By reading the traceback, we also know the exception-triggering function printdict() is invokved by indentprint(), and it is in turn invoked by printlist(), and so on.

If you run this in Jupyter notebook, the following is the output:

The information is essentially the same, but it gives you the lines before. and after each function call.

Triggering traceback manually

The easiest way to print a traceback is to add a raise statement to manually create an exception. But this will also terminate your program. If we want to print the stack at any time even without any exception, we can do so like the following:

The line traceback.print_stack() will print the current call stack.

But indeed, we often want to print the stack only when there is error (so we learn more about why it is so). The more common use case is the following:

This is a typical pattern for repeatedly calculating a function, such as Monte Carlo simulation. But if we are not careful enough, we may run into some error, such as in the above example, we may have division by zero. The problem is, in case of more complicated computation you can’t easily spot the flaw. Such as in above, the issue buried inside the call to compute(). Therefore it is helpful to understand how we get the error. But at the same time we want to handle the case of error rather than let the entire program terminate. If we use the try-catch construct, the traceback will not be print by default. Therefore we need the use the traceback.print_exc() statement to do it manually.

Actually we can have the traceback more elaborated. Because the traceback is the call stack and indeed we can examine each function in the call stack and check the variables in each level. In the complicated case, this is the function I usually use to do more detailed trace:

An example in model training

The call stack as reported in the traceback has a limitation: You can only see the Python functions. It should be just fine for the program you wrote but many large libraries in Python have part of them written in another language and compiled into binary. An example is Tensorflow. All the underlying operation are in binary for the performance. Hence if you run the following code, you will see something different:

The input_shape parameter to the first LSTM layer in the model should be (n_in, 1) to match the input data, rather than (n_in+1, 1). This code will print the following error once you invoked the last line:

If you look at the traceback, you cannot really see the complete call stack. For example, the top frame you know you called model.fit() but the second frame is from a function named error_handler(). Which you cannot see how the fit() function triggered that. This is because Tensorflow is highly optimized. A lot of stuff is hidden in compiled code and not visible by the Python interpreter.

In this case, it is essential to patiently read the traceback and find the clue to the cause. Of course, usually the error message should give you some useful hints as well.

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Books

Python Official Documentation

Summary

In this tutorial, you discovered how to read and print the traceback from a Python program.

Specifically, you learned:

  • What information the traceback tells you
  • How to print a traceback at any point of your program without raising an exception

In the next post, we will see how we can navigate the call stack inside the Python debugger.

2 Responses to Understanding Traceback in Python

  1. Manza Zityab December 31, 2021 at 4:09 pm #

    Thank you very much .I try to practice on this bash script . When I got some problem I will ask you dear .

    • James Carmichael January 1, 2022 at 12:11 pm #

      You are very welcome, Manza! Keep up the great work!

Leave a Reply