7  OILER Example #2

Let’s walk through another example of working with the OILER framework to solve a bug. This time, here’s our code and the error message that occurs when we try to run it:

Code 
def read_two_column_csv(file_name):
    with open(file_name, "r") as file:
        reader = csv.reader(file)
        next(reader)  # skip header row
        data = {}
        for item, value in reader:
            data[item] = value
    return data


def create_combined_inventory(quantities, prices):
    combined_data = {}
    for item in quantities:
        combined_data[item] = {"quantity": int(quantities[item]), "price": 0}
    for item in prices:
        price = float(prices[item].replace("$", ""))
        if item in combined_data:
            combined_data[item]["price"] = price
        else:
            combined_data[item] = {"price": price}
    return combined_data


## Process a shopping request
def check_one_product(inventory, product, request_qty):
    message = ""
    available_qty = inventory[product]["quantity"]
    price = inventory[product]["price"]

    if request_qty > available_qty:
        qty = available_qty
        apology = Sorry, we only have "
    else:
        qty = request_qty
        apology = ""
    subtotal = qty * price
    message = (
        f"${qty * price:.2f}: {apology}"
        f"{qty} {'unit' if qty == 1 else 'units'} of {product}; "
        f"${price:.2f} per unit.\n"
    )
    return subtotal, message


def process_shopping_request(shopping_request, inventory):
    response = ""
    total = 0
    for product, request_qty in shopping_request.items():
        subtotal, message = check_one_product(inventory, product, request_qty)
        total += subtotal
        response += message
    response += f"----\n${total:.2f} Total\n"
    return response


quantities = read_two_column_csv("data/quantities.csv")
prices = read_two_column_csv("data/prices.csv")

inventory = create_combined_inventory(quantities, prices)
print(process_shopping_request({"apples": 500, "bananas": 3, "pears": 1}, inventory))
print(process_shoping_request(
        {"apples": 500, "bananas": 3, "oranges": 4, "pears": 1}, inventory)
  Cell In[1], line 32
    apology = Sorry, we only have "
                                  ^
SyntaxError: unterminated string literal (detected at line 32)

7.1 Orient

Hopefully, the code looks extremely familiar now. As the functions and data types in play haven’t substantially changed from the last example, we’ll consider this step complete.

7.2 Investigate

7.2.1 Describe the symptom

The program has not fully executed and we have an error message. The error message says SyntaxError: unterminated string literal (detected at line 32).

7.2.2 Identify the proximal cause

7.2.2.1 Find the location

Reading the error message from the bottom up, we see that the proximate location is line 32, which is the line that instantiates the variable apology.

7.2.2.2 Hypothesize a cause

The error message is telling us there is an “unterminated string literal”, and the cart (^) symbol in the message is specifically pointing to the quotation mark in the expression apology = Sorry, we only have ".

“Unterminated” means something that has been started, but not properly concluded (if you didn’t know this word, searching the phrase “unterminated string literal” in your favorite search engine might help.). Indeed, we can see that the quotation mark in line 32 appears all alone–it has no match. Quotation marks used to declare the value of a string in Python need to come in pairs! We can therefore hypothesize that a quotation mark is missing to indicate the beginning of the string bound to the variable apology.

This seems like a clear case of a code vs. intention mismatch. The code that we actually wrote does not contain the correct syntax to accomplish what we intended.

7.3 Locate the root cause

Luckily, the root cause appears to be right in line 32. The root cause is the proximal cause (this will often happen for SyntaxError type bugs).

7.4 Experiment with fixes

A logical first guess would be to correct the syntax in line 32 by adding an opening quotation mark at the start of the string being specified: apology = "Sorry, we only have "

Let’s make this change and try rerunning the code.

Code 
def read_two_column_csv(file_name):
    with open(file_name, "r") as file:
        reader = csv.reader(file)
        next(reader)  # skip header row
        data = {}
        for item, value in reader:
            data[item] = value
    return data


def create_combined_inventory(quantities, prices):
    combined_data = {}
    for item in quantities:
        combined_data[item] = {"quantity": int(quantities[item]), "price": 0}
    for item in prices:
        price = float(prices[item].replace("$", ""))
        if item in combined_data:
            combined_data[item]["price"] = price
        else:
            combined_data[item] = {"price": price}
    return combined_data


## Process a shopping request
def check_one_product(inventory, product, request_qty):
    message = ""
    available_qty = inventory[product]["quantity"]
    price = inventory[product]["price"]

    if request_qty > available_qty:
        qty = available_qty
        apology = "Sorry, we only have "
    else:
        qty = request_qty
        apology = ""
    subtotal = qty * price
    message = (
        f"${qty * price:.2f}: {apology}"
        f"{qty} {'unit' if qty == 1 else 'units'} of {product}; "
        f"${price:.2f} per unit.\n"
    )
    return subtotal, message


def process_shopping_request(shopping_request, inventory):
    response = ""
    total = 0
    for product, request_qty in shopping_request.items():
        subtotal, message = check_one_product(inventory, product, request_qty)
        total += subtotal
        response += message
    response += f"----\n${total:.2f} Total\n"
    return response


quantities = read_two_column_csv("data/quantities.csv")
prices = read_two_column_csv("data/prices.csv")

inventory = create_combined_inventory(quantities, prices)
print(process_shopping_request({"apples": 500, "bananas": 3, "pears": 1}, inventory))
print(process_shoping_request(
        {"apples": 500, "bananas": 3, "oranges": 4, "pears": 1}, inventory))
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[4], line 56
     52     response += f"----\n${total:.2f} Total\n"
     53     return response
---> 56 quantities = read_two_column_csv("data/quantities.csv")
     57 prices = read_two_column_csv("data/prices.csv")
     59 inventory = create_combined_inventory(quantities, prices)

Cell In[4], line 3, in read_two_column_csv(file_name)
      1 def read_two_column_csv(file_name):
      2     with open(file_name, "r") as file:
----> 3         reader = csv.reader(file)
      4         next(reader)  # skip header row
      5         data = {}

NameError: name 'csv' is not defined

Interesting! Our program is still returning an error message. But it looks like a different one than before. This looks like a NameError- clearly, not the same SyntaxError that we just addressed in line 32.

It looks like we will have to troubleshoot this new error. Let’s go back in the OILER framework a few steps. We’re already oriented, so let’s return to the I stage.

7.5 Investigate (again)

7.5.1 Describe the symptom

Once again, the program has not fully executed. Now we have an error message with the keyword NameError, name 'csv' is not defined.

7.5.2 Identify the proximal cause

7.5.2.1 Find the location

What line does this occur in? The error message looks like it contains two line numbers, lines 56 and 3. But line 3 is the closest to the bottom of the error, meaning that line was attempted more recently in the program’s execution. Indeed, what we know from orienting ourselves to the code and looking at what lines 56 and 3 correspond to, we can see that line 56 invokes the function read_two_column_csv(). Line 3 is within this function. Specifically, this line contains the expression, reader = csv.reader(file).

7.5.2.2 Hypothesize a cause

A NameError typically occurs when “a local or global name is not found”, to quote the Python docs about built-in exceptions. The name mentioned in our error message is csv. What does that correspond to in line 3? It looks like csv is a module that should contain the function reader(), which we are invoking with the argument file.

One common reason that the name of a module will be undefined is that we forgot to import it. Checking the program, it’s clear we don’t begin by invoking import csv (and we’d usually expect import statements to be at the top of the script by convention). We can therefore hypothesize that we forgot to import csv before trying to use the csv module’s functions—another example of a code vs. intention mismatch!

7.6 Locate the root cause

The root cause here isn’t so much a line of code that is malformed, but a line that simply doesn’t exist. Our program should include an import csv statement at the very beginning, before our user-defined functions are created.

7.7 Experiment with fixes

The fix here should be to add import csv in line 1. Let’s do that and run the code again!

Code 
import csv

def read_two_column_csv(file_name):
    with open(file_name, "r") as file:
        reader = csv.reader(file)
        next(reader)  # skip header row
        data = {}
        for item, value in reader:
            data[item] = value
    return data


def create_combined_inventory(quantities, prices):
    combined_data = {}
    for item in quantities:
        combined_data[item] = {"quantity": int(quantities[item]), "price": 0}
    for item in prices:
        price = float(prices[item].replace("$", ""))
        if item in combined_data:
            combined_data[item]["price"] = price
        else:
            combined_data[item] = {"price": price}
    return combined_data


## Process a shopping request
def check_one_product(inventory, product, request_qty):
    message = ""
    available_qty = inventory[product]["quantity"]
    price = inventory[product]["price"]

    if request_qty > available_qty:
        qty = available_qty
        apology = "Sorry, we only have "
    else:
        qty = request_qty
        apology = ""
    subtotal = qty * price
    message = (
        f"${qty * price:.2f}: {apology}"
        f"{qty} {'unit' if qty == 1 else 'units'} of {product}; "
        f"${price:.2f} per unit.\n"
    )
    return subtotal, message


def process_shopping_request(shopping_request, inventory):
    response = ""
    total = 0
    for product, request_qty in shopping_request.items():
        subtotal, message = check_one_product(inventory, product, request_qty)
        total += subtotal
        response += message
    response += f"----\n${total:.2f} Total\n"
    return response


quantities = read_two_column_csv("data/quantities.csv")
prices = read_two_column_csv("data/prices.csv")

inventory = create_combined_inventory(quantities, prices)
print(process_shopping_request({"apples": 500, "bananas": 3, "pears": 1}, inventory))
print(process_shoping_request(
        {"apples": 500, "bananas": 3, "oranges": 4, "pears": 1}, inventory))
$10.00: Sorry, we only have 10 units of apples; $1.00 per unit.
$6.00: 3 units of bananas; $2.00 per unit.
$1.50: 1 unit of pears; $1.50 per unit.
----
$17.50 Total

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[5], line 63
     61 inventory = create_combined_inventory(quantities, prices)
     62 print(process_shopping_request({"apples": 500, "bananas": 3, "pears": 1}, inventory))
---> 63 print(process_shopping_requet(
     64         {"apples": 500, "bananas": 3, "oranges": 4, "pears": 1}, inventory))

NameError: name 'process_shopping_requet' is not defined

What’s this? Yet another error? Well at least it’s different! That means we are making some progress. Now, it appears our program is sailing through using the csv.reader() function…but it’s throwing an error at a different point in the execution flow. It looks like a problem in line 63, which is the second time we try to run the whole checkout process.

That’s somewhat reassuring, because we can run the whole checkout process at least once successfully (we did in line 62). But it appears to break the second time. Why?

Once again-again-let’s go back a few steps in the OILER process to I.

7.8 Investigate again, again

7.8.1 Describe the symptom

The program still has not fully executed and we have another error message with the type NameError. The message says, name 'process_shoping_request' is not defined.

7.8.2 Identify the proximal cose

7.8.2.1 Find the location

The line number indicated in the message is line 63, where we call the function that orchestrates the whole checkout process for a second time.

7.8.2.2 Hypothesize a cause

We know the checkout process ran with at least one set of groceries, because we did this in line 62. We know line 63 involves re-running the process with a different set of groceries, so maybe the problem is occurring because something is up with the new list of produce?

Then again, looking at the error message, the NameError is associated with the name ‘process_shoping_request’. That suggests the function isn’t defined–which we know it is, because we just used it in the previous line. One reason this could happen is if we misspelled the function in line 63. And indeed, if we look closely, there appears to be a typo! We thought we were calling process_shopping_request(), but we called process_shoping_request(). A minor difference of one letter ‘p’!

7.9 Locate the root cause

If our hypothesis is right, root cause is the same as the proximal cause–the typo in line 63.

7.10 Experiment with fixes

The natural fix would be to correct the typo. Let’s correct the function name to process_shopping_request() and try again.

Code 
import csv

def read_two_column_csv(file_name):
    with open(file_name, "r") as file:
        reader = csv.reader(file)
        next(reader)  # skip header row
        data = {}
        for item, value in reader:
            data[item] = value
    return data


def create_combined_inventory(quantities, prices):
    combined_data = {}
    for item in quantities:
        combined_data[item] = {"quantity": int(quantities[item]), "price": 0}
    for item in prices:
        price = float(prices[item].replace("$", ""))
        if item in combined_data:
            combined_data[item]["price"] = price
        else:
            combined_data[item] = {"price": price}
    return combined_data


## Process a shopping request
def check_one_product(inventory, product, request_qty):
    message = ""
    available_qty = inventory[product]["quantity"]
    price = inventory[product]["price"]

    if request_qty > available_qty:
        qty = available_qty
        apology = "Sorry, we only have "
    else:
        qty = request_qty
        apology = ""
    subtotal = qty * price
    message = (
        f"${qty * price:.2f}: {apology}"
        f"{qty} {'unit' if qty == 1 else 'units'} of {product}; "
        f"${price:.2f} per unit.\n"
    )
    return subtotal, message


def process_shopping_request(shopping_request, inventory):
    response = ""
    total = 0
    for product, request_qty in shopping_request.items():
        subtotal, message = check_one_product(inventory, product, request_qty)
        total += subtotal
        response += message
    response += f"----\n${total:.2f} Total\n"
    return response


quantities = read_two_column_csv("data/quantities.csv")
prices = read_two_column_csv("data/prices.csv")

inventory = create_combined_inventory(quantities, prices)
print(process_shopping_request({"apples": 500, "bananas": 3, "pears": 1}, inventory))
print(process_shopping_request(
        {"apples": 500, "bananas": 3, "oranges": 4, "pears": 1}, inventory))
$10.00: Sorry, we only have 10 units of apples; $1.00 per unit.
$6.00: 3 units of bananas; $2.00 per unit.
$1.50: 1 unit of pears; $1.50 per unit.
----
$17.50 Total

$10.00: Sorry, we only have 10 units of apples; $1.00 per unit.
$6.00: 3 units of bananas; $2.00 per unit.
$0.00: 4 units of oranges; $0.00 per unit.
$1.50: 1 unit of pears; $1.50 per unit.
----
$17.50 Total

Aha! It ran. Now, that doesn’t guarantee the program is entirely bug-free. We’d want to make sure the price calculations are correct, too, and maybe try to find some edge cases where the program isn’t as robust as we want. But we have made it through this battery of bugs.

7.11 Reflect

Now it’s time to reflect. Let’s make bug log entries for the three bugs we encountered.

7.11.1 Bug #1 log

  Cell In[1],   line 32
    apology = Sorry, we only have "
                                  ^
SyntaxError: unterminated string literal (detected at line 32)
Field Description
Symptom SyntaxError: unterminated string literal (detected at line 32)
Proximal Cause Code vs. intention mismatch: our code didn’t use the correct syntax to define a string.
Root Cause Same as the proximal cause
Fix When creating a string in Python, make sure all quotations come in pairs–one to begin and one to terminate the string sequence.

7.11.2 Bug #2 log

Cell In[4], line 3
      1 def read_two_column_csv(file_name):
      2     with open(file_name, "r") as file:
----> 3         reader = csv.reader(file)
      4         next(reader)  # skip header row
      5         data = {}

NameError: name 'csv' is not defined
Field Description
Symptom NameError: name ‘csv’ is not defined
Proximal Cause The csv module was not imported before attempting to use the function csv.reader().
Root Cause The program was missing the statement import csv, which should occur (along with any other import statements) at the beginning of the program. This could happen either because of a code vs. intention mismatch (we intended to use the csv module, but our code doesn’t actually import it) or because of a computation misunderstanding (if we did not understand that Python requires modules to be imported before use).
Fix Make sure all modules needed to execute the program have a corresponding import statement at the beginning of the program.

7.11.3 Bug #3 Log

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[5], line 63
     61 inventory = create_combined_inventory(quantities, prices)
     62 print(process_shopping_request({"apples": 500, "bananas": 3, "pears": 1}, inventory))
---> 63 print(process_shopping_requet(
     64         {"apples": 500, "bananas": 3, "oranges": 4, "pears": 1}, inventory))

NameError: name 'process_shopping_requet' is not defined
Field Description
Symptom NameError: name ‘process_shopping_requet’ is not defined
Proximal Cause Code vs. intention mismatch: there is a typo in line 63, where we misspelled the function process_shopping_request().
Root Cause This is the same as the proximal cause.
Fix Correct the typo in line 63 so we call the function by its correct name.

7.12 Revisiting our Bug Recipes

If we have not encountered similar bugs before, we might want to update our Bug Recipe book, too. There are several ways to organize your entries, but one logical way would be to make a Recipe for SyntaxError and NameError respectively (we can combine our learnings from Bugs #2 and #3, which were both NameError typed).

Bug Name Syntax Error
Symptoms Runtime error with message that mentions ‘SyntaxError’
Potential Proximal Causes Code vs. intention mismatch: a string may be improperly defined with only one quotation mark, instead of a pair of opening and closing quotation marks. This violates Python’s syntactic requirements.
Potential Root Causes Possibly the same as the proximal cause, if the line number indicated by the error message contains misused syntax.
Potential Fixes
  • Add the missing quotation mark.
  • Check Python documentation for working with strings to make sure you understand the syntax requirements for definining strings.
Bug Name NameError
Symptoms Runtime error with message that mentions ‘NameError’
Potential Proximal Causes
  • Attempting to use a module that has not been imported.
  • Misspelling in the name of a function.
Potential Root Causes
  • If the NameError is due to a typo in the line where execution stops, the root cause will be the same as the proximal cause.
  • If a function name is spelled as you intended in the line where execution stops, but the function definition contains a typo, the root cause could occur where the function is first created.
  • If an import statement is missing, the root cause is the absence of such a statement.
Potential Fixes
  • Check that function names are spelled correctly and uniformly everywhere in the program.
  • Check that every module used by your program has a corresponding import statement.

Finally, we could also consider any opportunities to clean up technical debt or improve documentation. In this case, though, the three bugs we encountered reveal more about our own programming practice than the design of the code. Particularly, they reveal the common tendency to make mistakes like typos! You’ll never stop making these kinds of errors, but you will get faster at spotting their patterns and resolving them. Keep putting in the time to learn from your bugs and your efficiency will increase with practice.