Experimenting with Python 1#: Custom types and exceptions

Being a developer in a big company and working with a system that is used by millions of people around a country sure is an interesting experience.

Even more when you like creating things and experimenting them on large scales.

I created a code generator for generating pages on an internal project at my team. I did it on the run so i did not had time to perfect it.

it was working, it was cool, but something wasn’t right.

The generator was not safe at all. When refactoring i had a really hard time figuring out what should go in or out of a method or component.

Not only that, but i had no idea of what kind of data was flowing around the generator components. Is it a code snippet? Is it a simple text? A key: value in text format?

So, since i was already building a new generator and most of the logic was already working (not flawless, but working) i thought about making some little experiments for testing a more readable way of passing values throughout the code using Python.

Heres what i found:

Hypothesis: Can objects serve as a way of enforcing a type of value?

In theory, an object can serve as a container for values.

There is a type conversion resource in almost every programming language. That means a value can be converted to another value, if compatible.

Take for example:

n1 = 8
n2 = str(8)
print(type(n1))
print(type(n2))
//<class 'int'>
//<class 'str'>

The conversion succeeded because the str() method has a set of requirements that needs to be fulfilled in order to convert the number 8 to the string ‘8’.

Now, lets look at another example:

n1 = 8
n2 = int('string')
//ValueError: invalid literal for int() with base 10: 'string'

The int() method threw an error because int() has a rule that in order to convert a text value to a number, that value needs to represent a number.

That way primitive types can enforce the integrity of the data being converted.

My hypothesis that we can use classes as a way of creating our own types in order to pinpoint the type of value that is required for our functions, and creating a foundation for proper error treatment.

Based in my observations and on previous experiments, a found out a type must contain:

• A set of requirements that needs to be met in order to properly assign the type value.
• A set of custom exceptions that later on can be used to treat irregular values.
• A set of permitted values.
• An easy way of reading the type value

The experiment:

In this experiment i will create a type that only accepts int numbers between 0 and 9.

I started by creating a class named singleDigitInt.

class singleDigitInt:

Validation:

I need a method that will validate the value:

def __validate(self):
    if type(self.value) != int: raise TypeError('the stored value is of incorrect type')
    if self.value > 9: raise numberTooHighError
    if self.value < 0: raise numberTooLowError

Code explanation:

if type(self.value) != int: raise TypeError

Here i am checking if the stored value is of int type.

if value > 9: raise numberTooHighError
if value < 0: raise numberTooLowError

It is throwing one error if the number is above 9 and another error if it is below 0.

You may be thinking: why didn’t i created a single test for those two scenarios?

It is because i want to handle high values in a different way low values would be handled.

Our validation is complete. But now here comes another problem: numberTooHighError and numberTooLowError are not base exceptions, they are custom exceptions.

So, lets implement it:

class numberTooHighError(Exception):
    def __init__(self, message = 'the number is above 9!'):
        self.message = message;
def __repr__(self):
        print(self.message)
class numberTooLowError(Exception):
    def __init__(self, message = 'the number is below 0!'):
        self.message = message;
def __repr__(self):
        print(self.message)

Those are a pretty straight-forward exceptions, it does not need to be complicated because the very fact of having a specific exception for an occasion can open the doors for specific treatment later on in different sections of the code.

Formatting:

Now i can implement the formatting of permitted values and edge-cases:

def __format(self, value):
    if type(value) == singleDigitInt: return value()
    if type(value) == int: return value
    result = value
    if type(result) == list and len(result) == 1: result = int(result[0])
    if type(_value) == str and \\
        re.search('b', _value) and \\
        re.search('[2-9]', _value) == None: _value = int(re.sub('b', '', _value), 2)
    return int(result)

Code explanation:

The formatting section needs to:

• Check if the passed value needs formatting
• Format permitted values and edge-cases

Checking if the passed value needs formatting:

I’ve identified two cases where formatting is not needed: When the value passed is already of type singleDigitInt and if the passed value is already an int number:

if type(value) == singleDigitInt: return value()

It’s was an excellent idea to verify if the value being passed is already of type singleDigitInt. If it is, it means that this value has already passed the necessary verifying and formatting. So there’s no need to format it again.

if type(value) == int: return value

If the passed value is an int number, it means it is already in the format that is possible for storing and verifying. Just return it.

Format permitted values and edge-cases:

Now we need to think: What are the edge-cases and possible values that a developer can use in the type and it should work?

I thought about:

• A one item long list
• A binary number

result = value

We start by cloning the value. Now i can use the modified value and the original value if needed

if type(_value) == list and len(_value) == 1: _value = _value[0]

Next i will check if the cloned value is a list, and if it is, if it have only one item.

The cloned value will not be converted to int just yet, because if someone feed this function an array with a binary number, we want that number to be converted to decimal.

if type(_value) == str and \\
    re.search('^b', _value) and \\
    re.search('[2-9]', _value) == None: _value = int(re.sub('b', '', _value), 2)

This is our first ‘complex’ test.

Here i imported the regex library and used it to verify if the value is made only of 0’s and 1’s by discarding any value that have numbers above 1.

Later on i found a bug where the string ‘10’ would be considered 2 by this system. So you need to flag if the number is binary by putting a b before the number.

That way if you numbers like 10, 11, 100, 101 or 111 will not be considered binaries, unless you explicitly want them to.

return int(result)

Finally, we return the value converted to int. If after the formatting the number still under 0 or above 10, it will be verified by the __validate() method later.

Retrieving the value:

Theres many ways we can easily retrieve a value from an object. Since we are using Python we will use the call method.

def __call__(self, *args, **kwargs):
        return self.__value

In a few words: the call method is something that will happen when the the object instance is called as a function.

If i do this:

x = singleDigitInt(4)
print(x())

The output will be 4.

Finished code:

Now i just need to finish the function with the init method:

def __init__(self, value):
        self.__value = self.__format(value);
        self.__validate()

And the end result will be:

import re
class singleDigitInt:
    def __init__(self, value):
        self.__value = self.__format(value);
        self.__validate()
    def __validate(self):
        if type(self.__value) != int: raise TypeError('the stored value is of incorrect type')
        if self.__value > 9: raise numberTooHighError
        if self.__value < 0: raise numberTooLowError
    def __format(self, value):
        if type(value) == singleDigitInt: return value()
        if type(value) == int: return value
        _value = value
        if type(_value) == list and len(_value) == 1: _value = _value[0]
        if type(_value) == str and \
                re.search('^b', _value) and \
                re.search('[2-9]', _value) == None: _value = int(re.sub('b', '', _value), 2)
        return int(_value)
    def __call__(self, *args, **kwargs):
        return self.__value
class numberTooHighError(Exception):
    def __init__(self, message = 'the number is too high!'):
        self.message = message;
    def __repr__(self):
        print(self.message)
class numberTooLowError(Exception):
    def __init__(self, message = 'the number is too low!'):
        self.message = message;
    def __repr__(self):
        print(self.message)

In this context, the __init__ method’s purpose is to determine which variables will be used throughout the type.

__values needed to be private because the last thing we want is someone instantiating the singleDigitInt type and immediately going number.value = ‘some random text’.

That would ruin all the purpose of the type. That way you are not submitting the value to checking or formatting.

Tests:

def test_simpleNumber(self):
    result = singleDigitInt(4)
    self.assertEqual(result(), 4)

This test is straight-forward. I give the class 4, it stores 4. So it works.

def test_arrayNumber(self):
    result = singleDigitInt([6])
    self.assertEqual(result(), 6)

This one is a permitted scenario. It receives a 1 item long array and stores the 0th item as a number. It works.

def test_binaryNumber(self):
    result = singleDigitInt('b1001')
    self.assertEqual(result(), 9)

Another permitted scenario. It receive a binary number, checks if it is a binary value, and if it is, converts it to a decimal number and stores it. Works as well.

def test_testNumberTooHigh(self):
    with self.assertRaises(numberTooHighError):
        singleDigitInt(20)
try:
        result = singleDigitInt(20)
    except numberTooHighError:
        result = singleDigitInt(9)
    self.assertEqual(result(), 9)

Here we are checking if the type throws an exception if the number is too high. If it does, the result will be the highest possible number. In this case: 9.

Let’s say you have a categorization data coming from somewhere, and those categories have ids. They are number between 0 and 9, being 5 the default value.

We can do this:

try:
    result = singleDigitInt(20)
except numberTooHighError:
    result = singleDigitInt(5)
self.assertEqual(result(), 5)

We can do the same with a number below zero:

def test_testNumberTooHigh(self):
    with self.assertRaises(numberTooLowError):
        singleDigitInt(-5)
try:
        result = singleDigitInt(-5)
    except numberTooLowError:
        result = singleDigitInt(0)
    self.assertEqual(result(), 0)

If the conversion failed because the number is too low, we can create another object with the smallest possible number.

This type of programming is case-specific, meaning the class will tell you what went wrong, in which checking it failed, and gives you the opportunity to treat it on the run.

A more robust test

Consider this case:

• I’m receiving an array of numbers coming from a server.
• This data has numbers that reflect a category.
• That server has legacy data. Those numbers can go from -1 to 15.
• They may or may not be a string.
• Recently the management reduced the number of categories to numbers between 0 and 9
• -1 categories are faulty data and they should not be considered.
• numbers above 9 will be included in category 9
• Oh… and theres the legacy part… when the database was made back in 1980, some numbers were stored in binary.
• Fortunately, the back-end guy changed all the categories IDs so they would have a ‘b’ before the binaries.

With my newly created type, i can do something like that:

def test_dataTest(self):
    dataFromDB = [0, -1, 5, 4, '11', 13, 15, '12', -1, 7, 14, '-1', -1, 2, 10, 'b01', 'b1000', 'b1111', 'b1010', 'b0100', '1']
    response = []
    for data in dataFromDB:
        try:
            newData = singleDigitInt(data)
        except numberTooHighError: newData = singleDigitInt(9)
        except numberTooLowError: continue
        response.append(newData())
    print(response)
# [0, 5, 4, 9, 9, 9, 9, 7, 9, 2, 9, 1, 8, 9, 9, 4, 1]

In conclusion:

So, in conclusion, what i observed using this method was:

Reusability:

To start, i can tell that if a number is too high or too low, it is not arbitrary or something that only this function have.

It is part of the singleDigitInt type.

Anywhere i use this class it will trigger those exceptions if the wrong values are provided.

This is good. Because it creates a centralized source of that value pattern. I do not need to worry about one function providing valid one digit numbers but another function not providing it.

Code communication:

When you work on large teams you will find a lot of different implementations for the same thing.

Creating type specific exceptions can make communication effective in your team:

• It can help on code reviews, since the developer reviewing the code can easily track the data flow inside the code.
• It can also help on patching bugs, since you can easily define an action to be taken when the value is faulty and communicate it.

Cleaner code:

Following the DRY principle, it would be a real problem if every time we need to use a single digit int we needed to verify all those things we checked above.

And by creating a method to make those checks, we do not have a signature for values coming from that method. So every time we want to ensure that kind of value, we would have to run that function again.

Having a signature for this kind of value is not only useful, but makes your code cleaner.

Hope you guys enjoyed this article. It is my first one and hopefully the first of many.

Thanks for reading!