5 Experiment with Fixes

At long last, it is time to try to fix the error. There will often be more than one approach for fixing the problem, just as there is often more than one way to implement a program in the first place. How to choose which to make is largely beyond the scope of this document. Here, we try to point you to potential fixes, and how to check whether they worked.

5.1 Strategy

Depending on the type of cause that you identified, there are different types of typical fixes. Some kinds of fixes have accompanying ways of verifying that the fix worked.

No matter what kind of fix you try, it will almost always be a good idea to some kind of local verification that the fix worked, rather than just running the whole program again. If you haven’t completely fixed the problem, there is a danger that the symptoms you originally observed will disappear, only to appear again in some future run. You’ve just done all the hard work of the Orient, Investigate, and Locate steps. You might as well reap the rewards of that hard work by making sure you have fixed the underlying cause.

Adding one ore more assert statements can be a particulary good way to verify your fix. In the Investigate step you may have written an assert that checked a hypothesis about some python expression evaluating to a bad value, now you will reverse that assertion to assert that the expression evaluates to a value that is what you intended.

There are two advantages of adding an assert statement to verify your fix:

It forces you to be clear about your expectations about the data.
If those expectations are ever violated on a future code run, perhaps with different inputs, you will be alerted about that expectation violation.

You’ll want to be judicious, however, about such added assert statements. Once you’ve checked that the assert passes for the data that triggered your original symptoms, if you’re pretty sure that it can never fail in the future, you may want to remove it, just so that your code doesn’t get too cluttered with extra assert statements that make it harder for people to read and understand.

5.1.1 Code vs. Intention Mismatch

If you diagnosed the cause as a mismatch between what your code tells the interpreter to do and what you intended it to do, it will usually be obvious how to fix it once you clearly understand the rules of python. The goal here is to “make it do what I meant”.

syntax and indentation errors
- add the missing colon, fix the mismatched parentheses, add or remove spaces, etc.
grouping errors
- add parentheses if there might be some ambiguity about how to group elements of complex expressions
wrong function or variable name
- replace with the intended variable or function
incorrectly accessing elements of a collection, like a list or dictionary
- change the expression that extracts a data object

After you have made a change, it will also be fairly straightforward to assess whether your fix has worked.

For syntax and spacing problems, color coding in your code editor will often help you see immediately whether you have well-formed python code.
For grouping errors, you can print the results of the complex expression with your newly added parentheses, or add an assert statement, .
Similarly, if you have adjusted an expression that extracts something from a complex data object, you can print out the value of that expression, or examine its value in an interactive debugger.
- This may be a good place to add an assert statement, even before you make the fix. Then you can try one or more fixes and have an automated test of whether your fix worked.

5.1.2 Computation Misunderstanding

If you misunderstood a computation that was being invoked, there are two possible ways to fix it:

provide the required input data. This may be relatively easy to do, if the required data object is already bound to a variable at that point in the execution, or extractable from some compound data object using an extraction expression. Sometimes, though, you will realize that the required data has not yet been computed. In that case, you will have to implement an additional computation, either at that point in the execution or somewhere earlier, to produce the data that is required at this point in the execution.
change the computation.
- If the code invokes a built-in function or library, there may be some other function that you can invoke that will operate on the data that you have available at this point in the execution. Or you may be able to write your function to substitute for it.
- If the code already invokes a user-defined function, you may be able to modify that function to operate on the data you have available. Be careful, however– if you modify the input parameters of the function, there may be invocations of the function that you will also need to update.
- In either case, you may also need to make changes to the rest of the code beyond this point in the execution. In an extreme case, you will completely rethink and replace the rest of the code.

5.1.3 Data Fault

If you have identified a data fault, a mismatch between expected and actual data, you can:

Change subsequent code to use the actual data. You may decide that the actual data provides a good basis for completing the program’s purpose, so long as you make some changes in the subsequent computations that are performed. You can try the same kinds of fixes as for changing the computation in response to a computation misunderstanding. That is, you can switch to invoking a different function, or you can change the implementation of the function. In the extreme, you may completely replace all of the code subsequent to the point in the execution where the data fault has been identified.
- If the actual data is missing something that the code expects, you could alter the processing code to check for that condition and take appropriate action for that case.
Change prior code to generate the expected data. Alternatively, you can make changes to the prior computations to produce the data that you need. This requires a design process of imagining how the available data could be transformed, through additional computation, to produce what is needed.
- If the expected data is missing, such as a missing key in a dictionary, your additional computation may explicitly add default values.
Change the initial data. If you identified a fault in the data that is initially input, you may decide to directly correct that. Since you will typically not control the input data for a program, it will typically be better not to manually edit the data. Instead, you can write code that performs pre-processing, checking for errors and filling in any missing data with default values.

5.2 Practice

Recall our running example. We identified a symptom, a runtime error on line TKTK. After investigation, we concluded that there was a data fault, that we were expecting to find a 'price' key in the inner dictionary inventory['oranges']. And we traced that back: the combine step never added that key, because it wasn’t in the prices dictionary. And it wasn’t in that dictionary because it wasn’t in the file prices.csv.

There is more than one option for how to fix this. In fact, we could fix it using all three of the generic approaches described above.

5.2.1 Change subsequent code to use the actual data

We could alter the processing code in check_one_product() to check for a missing price and handle it gracefully. Perhaps the store could decide that customers should benefit whenever the price is missing, by giving away the product for free. See the new lines 34 and 35 below.

Code

import csv


def read_two_column_csv(file_name):
    with open(file_name, "r") as file:
        reader = csv.reader(file)
        next(reader)  # skip header row
        data = {}
        for item, value in reader:
            data[item] = value
    return data


def create_combined_inventory(quantities, prices):
    combined_data = {}
    for item in quantities:
        combined_data[item] = {"quantity": int(quantities[item])}
    for item in prices:
        price = float(prices[item].replace("$", ""))
        if item in combined_data:
            combined_data[item]["price"] = price
        else:
            combined_data[item] = {"price": price}
    return combined_data


## Process a shopping request
def check_one_product(inventory, product, request_qty):
    message = ""
    available_qty = inventory[product]["quantity"]
    if "price" in inventory[product]:
        price = inventory[product]["price"]
    else:
        price = 0

    if request_qty > available_qty:
        qty = available_qty
        apology = "Sorry, we only have "
    else:
        qty = request_qty
        apology = ""
    subtotal = qty * price
    message = (
        f"${qty * price:.2f}: {apology}"
        f"{qty} {'unit' if qty == 1 else 'units'} of {product}; "
        f"${price:.2f} per unit.\n"
    )
    return subtotal, message


def process_shopping_request(shopping_request, inventory):
    response = ""
    total = 0
    for product, request_qty in shopping_request.items():
        subtotal, message = check_one_product(inventory, product, request_qty)
        total += subtotal
        response += message
    response += f"----\n${total:.2f} Total\n"
    return response


quantities = read_two_column_csv("data/quantities.csv")
prices = read_two_column_csv("data/prices.csv")

inventory = create_combined_inventory(quantities, prices)
print(process_shopping_request({"apples": 500, "bananas": 3, "pears": 1}, inventory))
print(
    process_shopping_request(
        {"apples": 500, "bananas": 3, "oranges": 4, "pears": 1}, inventory
    )
)

$10.00: Sorry, we only have 10 units of apples; $1.00 per unit.
$6.00: 3 units of bananas; $2.00 per unit.
$1.50: 1 unit of pears; $1.50 per unit.
----
$17.50 Total

$10.00: Sorry, we only have 10 units of apples; $1.00 per unit.
$6.00: 3 units of bananas; $2.00 per unit.
$0.00: 4 units of oranges; $0.00 per unit.
$1.50: 1 unit of pears; $1.50 per unit.
----
$17.50 Total

5.2.2 Change prior code to generate the expected data

Alternatively, we could alter the processing code in create_combined_inventory() to automatically fill in a price key even if none is set. See the new line 17 below.

Code

import csv


def read_two_column_csv(file_name):
    with open(file_name, "r") as file:
        reader = csv.reader(file)
        next(reader)  # skip header row
        data = {}
        for item, value in reader:
            data[item] = value
    return data


def create_combined_inventory(quantities, prices):
    combined_data = {}
    for item in quantities:
        combined_data[item] = {"quantity": int(quantities[item]), "price": 0}
    for item in prices:
        price = float(prices[item].replace("$", ""))
        if item in combined_data:
            combined_data[item]["price"] = price
        else:
            combined_data[item] = {"price": price}
    return combined_data


## Process a shopping request
def check_one_product(inventory, product, request_qty):
    message = ""
    available_qty = inventory[product]["quantity"]
    price = inventory[product]["price"]

    if request_qty > available_qty:
        qty = available_qty
        apology = "Sorry, we only have "
    else:
        qty = request_qty
        apology = ""
    subtotal = qty * price
    message = (
        f"${qty * price:.2f}: {apology}"
        f"{qty} {'unit' if qty == 1 else 'units'} of {product}; "
        f"${price:.2f} per unit.\n"
    )
    return subtotal, message


def process_shopping_request(shopping_request, inventory):
    response = ""
    total = 0
    for product, request_qty in shopping_request.items():
        subtotal, message = check_one_product(inventory, product, request_qty)
        total += subtotal
        response += message
    response += f"----\n${total:.2f} Total\n"
    return response


quantities = read_two_column_csv("data/quantities.csv")
prices = read_two_column_csv("data/prices.csv")

inventory = create_combined_inventory(quantities, prices)
print(process_shopping_request({"apples": 500, "bananas": 3, "pears": 1}, inventory))
print(
    process_shopping_request(
        {"apples": 500, "bananas": 3, "oranges": 4, "pears": 1}, inventory
    )
)

$10.00: Sorry, we only have 10 units of apples; $1.00 per unit.
$6.00: 3 units of bananas; $2.00 per unit.
$1.50: 1 unit of pears; $1.50 per unit.
----
$17.50 Total

$10.00: Sorry, we only have 10 units of apples; $1.00 per unit.
$6.00: 3 units of bananas; $2.00 per unit.
$0.00: 4 units of oranges; $0.00 per unit.
$1.50: 1 unit of pears; $1.50 per unit.
----
$17.50 Total

5.2.3 Change the initial data

As a final alternative, we could pre-process the initial .csv file to add a default price for any product that is missing one. See the new function on lines 3-24, its invocation on line 84, and the use of the new file prices_with_defaults.csv on line 86.

Code

import csv


def preprocess_prices(old_prices_fname, inventory_fname, new_prices_fname):
    # Read the inventory to find all the products
    # Read the prices to find the prices of the products; if a product is not in the prices file, set the price to 0 and rewrite the file

    with open(inventory_fname, "r") as file:
        reader = csv.reader(file)
        next(reader)

        products = [item for item, _ in reader]

    with open(old_prices_fname, "r") as file:
        reader = csv.reader(file)
        next(reader)
        prices = {item: price for item, price in reader}

    with open(new_prices_fname, "w") as file:
        writer = csv.writer(file)
        writer.writerow(["Name", "Price"])
        for product in products:
            if product not in prices:
                prices[product] = "$0.00"
            writer.writerow([product, prices[product]])


def read_two_column_csv(file_name):
    with open(file_name, "r") as file:
        reader = csv.reader(file)
        next(reader)  # skip header row
        data = {}
        for item, value in reader:
            data[item] = value
    return data


def create_combined_inventory(quantities, prices):
    combined_data = {}
    for item in quantities:
        combined_data[item] = {"quantity": int(quantities[item])}
    for item in prices:
        price = float(prices[item].replace("$", ""))
        if item in combined_data:
            combined_data[item]["price"] = price
        else:
            combined_data[item] = {"price": price}
    return combined_data


## Process a shopping request
def check_one_product(inventory, product, request_qty):
    message = ""
    available_qty = inventory[product]["quantity"]
    print(f"For {product} inventory, keys are {list(inventory[product].keys())}")
    price = inventory[product]["price"]

    if request_qty > available_qty:
        qty = available_qty
        apology = "Sorry, we only have "
    else:
        qty = request_qty
        apology = ""
    subtotal = qty * price
    message = (
        f"${qty * price:.2f}: {apology}"
        f"{qty} {'unit' if qty == 1 else 'units'} of {product}; "
        f"${price:.2f} per unit.\n"
    )
    return subtotal, message


def process_shopping_request(shopping_request, inventory):
    response = ""
    total = 0
    for product, request_qty in shopping_request.items():
        subtotal, message = check_one_product(inventory, product, request_qty)
        total += subtotal
        response += message
    response += f"----\n${total:.2f} Total\n"
    return response


preprocess_prices(
    "data/prices.csv", "data/quantities.csv", "data/preprocessed_prices.csv"
)
quantities = read_two_column_csv("data/quantities.csv")
prices = read_two_column_csv("data/preprocessed_prices.csv")

inventory = create_combined_inventory(quantities, prices)
print(process_shopping_request({"apples": 500, "bananas": 3, "pears": 1}, inventory))
print(
    process_shopping_request(
        {"apples": 500, "bananas": 3, "oranges": 4, "pears": 1}, inventory
    )
)

For apples inventory, keys are ['quantity', 'price']
For bananas inventory, keys are ['quantity', 'price']
For pears inventory, keys are ['quantity', 'price']
$10.00: Sorry, we only have 10 units of apples; $1.00 per unit.
$6.00: 3 units of bananas; $2.00 per unit.
$1.50: 1 unit of pears; $1.50 per unit.
----
$17.50 Total

For apples inventory, keys are ['quantity', 'price']
For bananas inventory, keys are ['quantity', 'price']
For oranges inventory, keys are ['quantity', 'price']
For pears inventory, keys are ['quantity', 'price']
$10.00: Sorry, we only have 10 units of apples; $1.00 per unit.
$6.00: 3 units of bananas; $2.00 per unit.
$0.00: 4 units of oranges; $0.00 per unit.
$1.50: 1 unit of pears; $1.50 per unit.
----
$17.50 Total