Exceptions - Following a traceback#

This workshop will help you understand how to read and understand error messages in python.

Learning Goals#

  • Understand what Exceptions are and differentiate from the traceback

  • Read and interpret a traceback for easier debugging

  • When/how to use Exceptions in my own code

  • Seeking help when encountering an error

Terminology#

  • Error/Exception: A value and/or message informing a user that something unexpected has occurred and that the script should not continue running.

  • Traceback: A traceback is a message that points to where in the code an error has occurred.

Note

Exceptions tell you what went wrong, tracebacks tell you where it went wrong

parts of an exception

Common exceptions and common causes#

Exception

Interpretation

Generic Solution

SyntaxError
docs

Python can not read some of the code you wrote

Review your code and make sure you have no open quotes, brackets, parentheses. If you ever see a SyntaxError traceback where nothing looks wrong, check the line of code above it.

TypeError
docs

You’ve called a function/method and one of your arguments is of the wrong type

Check the types of each argument that you’re passing into the function in question. Ensure each argument you’re passing is appropriate for the function at hand

ValueError
docs

You’ve provided a value into a function that it cannot use

Check the function you’re calling, if it requires specific values (e.g. a positive number, a limited number of choices of strings, etc) make sure that your argument falls within those values

IndexError
docs

The object you are trying to index is either not indexable, or does not have an index corresponding to your selection

Double check the index number you are using to index and ensure it is not greater than the length of the container object.

ModeuleNotFoundError docs

You’re trying to import a package without having installed it first

Ensure you do not have a typo in the package name at import statement (variable names are case sensitive too!). If there are no typos, check your package managers (typically pip or conda) to ensure you have the specific package installed. If you are using a virtual environment manager, check that the correct virtual environment is active.

NameError docs

You are attempting to reference a variable that does not exist.

Ensure you have spelled your variable name correctly. Additionally, if you are working within a notebook, double check that have run the specific cell that defines your variable.

AttributeError

You attempted to access an attribute of an object that does not exist

You trying to access an attribute on an object that does not exist

SyntaxError#

Python can’t parse/read some of the code you wrote

Solution: Review your code and make sure you have no open quotes, brackets, parentheses

# Missing closing quote
a = "1
  Input In [1]
    a = "1
        ^
SyntaxError: unterminated string literal (detected at line 2)
# Missing closing parentheses
a = (1, 2
  Input In [2]
    a = (1, 2
        ^
SyntaxError: '(' was never closed
# Missing colon
for i in range(5)
    pass
  Input In [3]
    for i in range(5)
                     ^
SyntaxError: expected ':'

Less helpful SyntaxErrors#

Sometimes python’s SyntaxErrors aren’t exactly helpful…

Solution: If you ever see a SyntaxError traceback where nothing looks wrong, check the line of code above it.

# The traceback indicates an issue on line 3, 
#    whereas the issue is actually on line 2

x = (1, 2,
y = 3
  Input In [4]
    x = (1, 2,
        ^
SyntaxError: '(' was never closed
# The traceback indicates an issue on line 5, 
#     whereas the issue is actually on line 4

my_dict = {
    "hello": 1,
    "world": 3
    "foo": 4,
    "bar": 5
}
  Input In [5]
    "world": 3
             ^
SyntaxError: invalid syntax. Perhaps you forgot a comma?

TypeError#

A function/method was given an argument of the wrong type

Solution: Check the types of each argument that you’re passing into the function in question. Ensure each argument you’re passing is appropriate for the function at hand

integer_list = [1, 2, 3]
sum(integer_list)
6
string_list = ["a", "b", "c", "d"]
sum(string_list)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [7], in <cell line: 2>()
      1 string_list = ["a", "b", "c", "d"]
----> 2 sum(string_list)

TypeError: unsupported operand type(s) for +: 'int' and 'str'

TypeError - Not callable#

You are attempting to call a function, however the object you are using is not a function*

What is function calling? Function calling is when you invoke a function by using parentheses.

  • Note: more specifically, the object you are trying to call does not define a __call__ dunder method, which is not limited to just function objects.

# Defining the function object
def addition(x, y):
    return x + y

addition(1, 2) # I'm now calling the function, works as expected
3
# Whoops, we overwrote the function name with a variable
#    that points to an integer

addition = 5
addition(1, 2) 
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [9], in <cell line: 5>()
      1 # Whoops, we overwrote the function name with a variable
      2 #    that points to an integer
      4 addition = 5
----> 5 addition(1, 2)

TypeError: 'int' object is not callable

addition no longer references a function and instead points to an integer. An integer is not callable (it doesn’t define how to handle opening/closing parentheses) so an error is raised

ValueError#

You’ve provided a value into a function that it cannot use

Similar to a TypeError in terms of its logic (e.g. ‘this function received something it doesn’t know how to work with), but differentiates in that this is used to raise an Error when the type of the passed argument is correct.

Solution: Check the function you’re calling, if it requires specific values (e.g. a positive number, a limited number of choices of strings, etc) make sure that your argument falls within those values

import math
math.sqrt(100) # works as expected
10.0
# does not return imaginary numbers, 
#    instead raises an Exception
math.sqrt(-100) 
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [11], in <cell line: 3>()
      1 # does not return imaginary numbers, 
      2 #    instead raises an Exception
----> 3 math.sqrt(-100)

ValueError: math domain error
# For comparison, this snippet raises a TypeError
#   note that we're passing a string into `math.sqrt`
math.sqrt("100")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [12], in <cell line: 3>()
      1 # For comparison, this snippet raises a TypeError
      2 #   note that we're passing a string into `math.sqrt`
----> 3 math.sqrt("100")

TypeError: must be real number, not str

IndexError#

The object you are trying to index is either not indexable, or does not have an index corresponding to your selection.

These are common to encounter with container-like objects. (e.g. lists, tuples, sets, arrays)

Solution: Double check the index number you are using to index and ensure it is not greater than the length of the container object.

# successful indexing
x = ["a", "b", "c"]
x[1]  # get the second item from the list
'b'
x[10] # get the 10th item from the list
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Input In [14], in <cell line: 1>()
----> 1 x[10]

IndexError: list index out of range
# A method can also raise an IndexError if that method attempts 
#     to index an item from the object
x.pop(10)
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Input In [15], in <cell line: 3>()
      1 # A method can also raise an IndexError if that method attempts 
      2 #     to index an item from the object
----> 3 x.pop(10)

IndexError: pop index out of range

ModuleNotFoundError#

You’re trying to import a package without having installed it first

Solution: Ensure you do not have a typo in the package name at import statement (variable names are case sensitive too!). If there are no typos, check your package managers (typically pip or conda to ensure you have the specific package installed. If you are using a virtual environment manager, check that the correct virtual environment is active.

# runs successfully, as expected
import collections
# error, we have not installed a package named `does_not_exist`
import does_not_exist
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Input In [17], in <cell line: 2>()
      1 # error, we have not installed a package named `does_not_exist`
----> 2 import does_not_exist

ModuleNotFoundError: No module named 'does_not_exist'
# common: watch out for typos!
import colections
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Input In [18], in <cell line: 2>()
      1 # common: watch out for typos!
----> 2 import colections

ModuleNotFoundError: No module named 'colections'

NameError#

You are attempting to reference a variable that does not exist

Solution: Ensure you have spelled your variable name correctly. Additionally, if you are working within a notebook, double check that have run the specific cell that defines your variable.

memoization = 0
print(memoization)
0
print(memoizaton)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Input In [20], in <cell line: 1>()
----> 1 print(memoizaton)

NameError: name 'memoizaton' is not defined

AttributeError#

You trying to access an attribute on an object that does not exist

Solution:

  1. Check you have correctly spelled the attribute

  2. Check the object you are operating, to ensure it is what you think it is.

import datetime as dt
todays_date = dt.datetime.today()
print(todays_date)
2024-11-06 14:15:54.476045
# Accessing the year attribute from the datetime object
todays_date.year
2024
# Misspelled attribute raises AttributeError
todays_date.yer
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [23], in <cell line: 2>()
      1 # Misspelled attribute raises AttributeError
----> 2 todays_date.yer

AttributeError: 'datetime.datetime' object has no attribute 'yer'
# Accidental overitten variable raises AttributeError
todays_date = 5
todays_date.year  # `todays_date` is no longer a datetime object
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [24], in <cell line: 3>()
      1 # Accidental overitten variable raises AttributeError
      2 todays_date = 5
----> 3 todays_date.year

AttributeError: 'int' object has no attribute 'year'

Tracebacks From Our Functions#

So far we have only discussed errors where the only code in the traceback is code we’ve written. However, code is never this simple. Lets take a look at more realistic scenarios where errors are encountered within functions and generate much longer tracebacks that include a lot of code we haven’t seen before.

It is common practice for functions to be nested- meaning that functions are called inside of functions. For each level of nesting, we will see a slightly longer traceback.

def add(x, y):
    return x + y

add("2", 1)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [25], in <cell line: 4>()
      1 def add(x, y):
      2     return x + y
----> 4 add("2", 1)

Input In [25], in add(x, y)
      1 def add(x, y):
----> 2     return x + y

TypeError: can only concatenate str (not "int") to str
def add_one(x):
    return add(x, 1)

add_one("2")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [26], in <cell line: 4>()
      1 def add_one(x):
      2     return add(x, 1)
----> 4 add_one("2")

Input In [26], in add_one(x)
      1 def add_one(x):
----> 2     return add(x, 1)

Input In [25], in add(x, y)
      1 def add(x, y):
----> 2     return x + y

TypeError: can only concatenate str (not "int") to str

Note

The traceback became longer when we called add from within add_one because python wants to show us all of the code that was executed between the code we wrote, and where the error occurred.

Encountering nested functions like this, is extremely common. So it’s important that you can follow these longer tracebacks to help you debug your own code.

Lets look at example where we import an external library and encounter an error in code we haven’t written ourselves.

Breakdown a Nested Traceback#

Using the example above, lets follow the execution path that led to the error.

This help us understand what function calls were made that led to the raising of the error

follow a traceback

Tracebacks From External Functions#

It is common practice to use external libraries in python. When doing so you have access to tons of code that other python users have written. Our tracebacks can become a little funny looking because now we end up seeing code that we have neither written, nor read before. It is important to be able to follow the execution path of code even when you did not write all of the code that is in a given traceback.

import collections

# A Counter object returns a dictionary of unique values along with how many times they were observed
counter = collections.Counter("aaaabbbccd")
counter
Counter({'a': 4, 'b': 3, 'c': 2, 'd': 1})
# This Counter object has a few more methods than a normal dictionary
#  To get the 2 most commonly encountered values:
counter.most_common(2)
[('a', 4), ('b', 3)]
# What happens if we put a string into the most_common method?
counter.most_common("a")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [29], in <cell line: 2>()
      1 # What happens if we put a string into the most_common method?
----> 2 counter.most_common("a")

File ~/.pyenv/versions/3.10.2/lib/python3.10/collections/__init__.py:602, in Counter.most_common(self, n)
    600 # Lazy import to speedup Python startup time
    601 import heapq
--> 602 return heapq.nlargest(n, self.items(), key=_itemgetter(1))

File ~/.pyenv/versions/3.10.2/lib/python3.10/heapq.py:540, in nlargest(n, iterable, key)
    538     pass
    539 else:
--> 540     if n >= size:
    541         return sorted(iterable, key=key, reverse=True)[:n]
    543 # When key is none, use simpler decoration

TypeError: '>=' not supported between instances of 'str' and 'int'

Breakdown an External Traceback#

follow external traceback

Using Errors in your own code#

Sometimes it is necessary to raise errors in your own code. Whether you want to limit the range of possible inputs for a function, or want to avoid catastrophic failure of your code, errors are a useful way of conveying to a user that something has gone wrong.

Raise your own errors#

# Errors can be made, just like any other object in python
my_error = ValueError("this is a message")
my_error
ValueError('this is a message')

Note

With the above code, the error did not do anything. We simply created an error object. In order to have the error object stop python from executing code, we will need to raise it.

# Errors can be throw by using the `raise` statement
raise my_error
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [31], in <cell line: 2>()
      1 # Errors can be throw by using the `raise` statement
----> 2 raise my_error

ValueError: this is a message

Note

Both the error type and message we wrote ourselves was included in the output!

Now lets explore some occasions that we may want to throw our own errors to prevent users from misusing our functions.

Raise Your Own Errors - Example#

def add_then_multiply(x, y, multiplier=1):
    return (x + y) * multiplier

# adds 3 + 5, multiplies the result by 2
add_then_multiply(4, 5, multiplier=2)
18

Our add_then_multiply function works as expected. First we add 4 + 5 together. Then we multiply their sum by 2.

Let’s see what happens when we change the multiplier argument from an number (integer/float) to a string.

add_then_multiply(4, 5, multiplier="2")
'222222222'

Well that was unexpected, the function still works despite our multiplier value being a string, however, the output is very odd though- a repeating string of “2”s?

Let’s see if we can use our own Exception to ensure only numbers can be passed into our function. We’re going to use a TypeError when an inappropriate type of value is passed into the multiplier argument (e.g. any value that is not an integer or float).

def add_then_multiply(x, y, multiplier=1):
    if not isinstance(multiplier, (int, float)):
        raise TypeError("multiplier must be an integer or float")
    return (x + y) * multiplier

add_then_multiply(3, 5, multiplier="2")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [34], in <cell line: 6>()
      3         raise TypeError("multiplier must be an integer or float")
      4     return (x + y) * multiplier
----> 6 add_then_multiply(3, 5, multiplier="2")

Input In [34], in add_then_multiply(x, y, multiplier)
      1 def add_then_multiply(x, y, multiplier=1):
      2     if not isinstance(multiplier, (int, float)):
----> 3         raise TypeError("multiplier must be an integer or float")
      4     return (x + y) * multiplier

TypeError: multiplier must be an integer or float

Now instead of having an unexpected result, we raise an informative error indicating that this function is not supposed to receive a non-number for its multiplier argument.

Handling Errors - try, except, else, and finally#

Sometimes you expect to receive an error and want to run a snippet of code in order to handle that error.

The try…except code-block has many more features than described here, however they are outside the scope of this article so we won’t get into them here.

# Any set up code should exist boutside of the try block.
#  In general, the try block should not contain many lines of code.
#  Just the lines of code that raise an Exception

x = [1, 2, 3]

try:
    # The code that may raise an error should occur here.
    x[10] # this will raise an IndexError
except IndexError:
    # This block will be executed if an IndexError occurred in the `try` block
    #   if this block does not explicitly raise an error, the error can be ignored.
    print("There was an error!")
else:
    # This block is only executed if NO error has occurred
    print("no error occurred")
finally:
    # anything in this block will always be executed, even if an error is encoutnered
    print("I will always be executed")
There was an error!
I will always be executed

Challenge! - See if you can fix these errors#

# Challenge 1 - fix the import statement
improt collections
  Input In [36]
    improt collections
           ^
SyntaxError: invalid syntax
# Challenge 2 - pop the last element of the list
x = ["a", "b", "c"]
x.pop(3)
print(x)
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Input In [37], in <cell line: 3>()
      1 # Challenge 2 - pop the last element of the list
      2 x = ["a", "b", "c"]
----> 3 x.pop(3)
      4 print(x)

IndexError: pop index out of range
# Challenge 3 - Fix the SyntaxError
if True:
    if True:
        print(((123))

if not False:
    print(456)
  Input In [38]
    print(((123))
         ^
SyntaxError: '(' was never closed
# Challenge 4 - Fix the SyntaxError
for i in range(3):
    print(("loop counter", i)
    if i <= 1
        print("inside if statement:", i)
  Input In [39]
    print(("loop counter", i)
          ^
SyntaxError: expected 'else' after 'if' expression
# Challenge 5 - Write your own code to make the longest traceback possible!

Bonus! Errors in Python >= 3.10#

Python 3.10 (which is currently not released at the time of writing) has revamped the usefulness of tracebacks! The new Error messages are much more specific, especially when it comes to syntax errors.

https://realpython.com/lessons/better-error-messages/