Introduction To Python Unit Testing

What Is Unit Test?

Unit Tests are software programs written to exercise other software programs (called Code Under Test) with specific preconditions and verify the expected behaviours of the CUT.

Unit tests are usually written in the same programming language as their code under test.

Each unit test should be small and test only limited piece of code functionality. Test cases are often grouped into Test Groups or Test Suites. There are many open source unit test frameworks. The popular ones usually follows an xUnit pattern invented by Kent Beck E.g. JUnit for Java, CppUTest for C/C++.

Unit tests should also run very fast. Usually we expect to run hundreds of unit test cases within a few seconds.

Think About It

Why are unit tests usually written in the same programming language as the code under test?

A Simple Example

We use Python's standard unittest module (was called PyUnit) as the example here.

In [1]: 
%%file testwrap.py
import unittest #& yellow
from textwrap import TextWrapper

class TestForTextWrapping(unittest.TestCase): #& yellow
    
    def test_no_wrapping_when_string_length_is_less_than_line_width(self): #& yellow
        wrapper = TextWrapper(width=10)
        wrapped = wrapper.wrap("a" * 5)
        self.assertEqual(["a" * 5], wrapped) #& yellow

if __name__ == '__main__':
    unittest.main()

Writing testwrap.py

This unit test tests the TextWrapper class from the standard Python module textwrap. Of course usually it doesn't make sense to test the standard modules. Those modules have their own tests. Here we just use it as an example for how to write unit test in Python.

Basic elements of a Python unit test

Creating a Python unit test is easy.

  • import the unittest module, or TestCase from the unittest module.
  • Inherit from the unittest.TestCase class. The derived class is a test group (Yes, the names are a bit confusing).
  • Any function name that starts with test_ will be considered as a test case in the test group.
  • Inside the test, there are potentially several assertion method calls that are used to check the expected result, e.g. assertEqual, assertTrue, assertFalse.

Usually, in an xUnit framework, for an assertion with 2 parameters, the convention is that the 1st argument is the expected result and the second argument is the actual result. But somehow in python unittest module this is not emphasized.

To run unit test cases you need to call unittest.main(). Then the unittest framework will run through all the unit tests it can find. In the above example, we already put it in the same .py file. So we can simply run it with the

python -munittest <test_module_name>

command, or

python <test_module_name>.py

(since we added unittest.main() in that module)

In [2]: 
!python testwrap.py

.
----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

No news is good news

If the test passes, it should just print OK (and perhaps some dots to show the progress). No other information.

Rule of thumb:

No human intervention should be needed, either to get ready for the test, running the test cases, or checking the result.

Adding a failing test case.

In [3]: 
%%file testwrap.py
import unittest
from textwrap import TextWrapper

class TestForTextWrapping(unittest.TestCase):
    
    def test_no_wrapping_when_string_length_is_less_than_line_width(self):
        wrapper = TextWrapper(width=10)
        wrapped = wrapper.wrap("a" * 5)
        self.assertEqual(["a" * 5], wrapped)
    
    def test_no_wrapping_when_string_length_is_more_than_line_width(self): #& white
        wrapper = TextWrapper(width=10)                                    #& white
        wrapped = wrapper.wrap("aaaaa bbbbb")                              #& white
        self.assertEqual(["aaaaa bbbbb"], wrapped)                         #& white

if __name__ == '__main__':
    unittest.main()

Overwriting testwrap.py

Run it again. It should give precise information if any test fails.

In [4]: 
!python -munittest testwrap

.F
======================================================================
FAIL: test_no_wrapping_when_string_length_is_more_than_line_width (testwrap.TestForTextWrapping)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "testwrap.py", line 14, in test_no_wrapping_when_string_length_is_more_than_line_width
    self.assertEqual(["aaaaa bbbbb"], wrapped)
AssertionError: Lists differ: ['aaaaa bbbbb'] != ['aaaaa', 'bbbbb']

First differing element 0:
aaaaa bbbbb
aaaaa

Second list contains 1 additional elements.
First extra element 1:
bbbbb

- ['aaaaa bbbbb']
?        ^

+ ['aaaaa', 'bbbbb']
?        ^^^^


----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1)

Think About It

Which of the following does not need to be automated in a unit test case?

  1. Setup (creating the environment and precondition)
  2. Verification (I can check the result manually)
  3. Partial verification
  4. Recovering (or Teardown)
  5. Unit test need to be fully automated, repeatable and self-verified.

Fix the test now.

In [5]: 
%%file testwrap.py
import unittest
from textwrap import TextWrapper

class TestForTextWrapping(unittest.TestCase):
    
    def test_no_wrapping_when_string_length_is_less_than_line_width(self):
        wrapper = TextWrapper(width=10)
        wrapped = wrapper.wrap("a" * 5)
        self.assertEqual(["a" * 5], wrapped)
    
    def test_text_should_wrap_when_string_length_is_more_than_line_width(self):
        wrapper = TextWrapper(width=10)
        wrapped = wrapper.wrap("aaaaa bbbbb")
        self.assertEqual(["aaaaa", "bbbbb"], wrapped)

if __name__ == '__main__':
    unittest.main()

Overwriting testwrap.py

Unit test should run very fast

In [6]: 
!time python -munittest testwrap

..
----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

real	0m0.050s
user	0m0.031s
sys	0m0.016s

You should be able to run hundreds of unit tests within seconds.

Think About It

In order to keep a unit test fast, which of the following do you need to avoid?

  1. Using the network
  2. Using the file system
  3. Connecting to a database
  4. Using a slow collaborator directly in the code under test
  5. all of above

Think About It

If your unit tests:

  • Needs little intervention (you just need to trigger it)
  • Runs very fast (less than 3 seconds)

How often could you execute your unit tests?

  1. Only run it on my Continuous Integration server
  2. Run it once per day
  3. Every hour
  4. 10 times per hour
  5. Every minute. It frees me from worrying about making trivial mistakes. I can focus on more important stuff.

Test Cases Discovery

In the example above, we have only one test module. But quite often we will need more test modules. If you still want to run the unit test in the same way, you need to manually import all the test modules into one module.

There are several ways to discover test cases automatically. I would recommend nosetest at the moment. To install it:

pip install nose

or

easy_install nose

Then you can simply run:

In [7]: 
!nosetests

..
----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

Test setUp and tearDown

Test group can have two optional functions, setUp and tearDown. setUp will be called before each test case. tearDown will be called after each test case. tearDown will be called independently from the successful (or not) test outcome. It will be called even when there is an un-captured exception during the test case.

Test setup and teardown are sometimes called test fixtures. They are meant to create specific conditions of the code under test for the test cases. But a much easier way to understand their purpose is to view them as remove code duplicates among the test cases in the same test group.

Our previous tests can be changed to:

In [8]: 
%%file testwrap.py
import unittest
from textwrap import TextWrapper

class TestForTextWrapping(unittest.TestCase):
    
    def setUp(self):
        self.wrapper = TextWrapper(width=10)
        
    def test_no_wrapping_when_string_length_is_less_than_line_width(self):
        wrapped = self.wrapper.wrap("a" * 5)
        self.assertEqual(["a" * 5], wrapped)
    
    def test_text_should_wrap_when_string_length_is_more_than_line_width(self):
        wrapped = self.wrapper.wrap("aaaaa bbbbb")
        self.assertEqual(["aaaaa", "bbbbb"], wrapped)

if __name__ == '__main__':
    unittest.main()

Overwriting testwrap.py
In [9]: 
!nosetests

..
----------------------------------------------------------------------
Ran 2 tests in 0.002s

OK

Use test fixtures to get rid of duplicate code in the test cases.

Test cases using the same test fixtures should be grouped together.

Good Unit Test Pattern

Arrange, Act, Assert

A good pattern to follow in a unit test is "AAA": Arrange, Act and Assert.

If you can easily find this pattern in each of your test cases, your tests should be easy to understand, and they should be fairly specific.

In [10]: 
import unittest
class TestGroupForTextWrapping(unittest.TestCase):
    
    def test_should_have_no_wrapping_when_string_length_is_5_and_line_width_is_10(self):
        # Arrange:  Arrange all necessary preconditions and inputs. 
        wrapper = TextWrapper(width=10)
        
        # Act:  Act on the object or method under test. 
        wrapped = wrapper.wrap("a" * 5)
        
        # Assert:  Assert that the expected results have occurred. 
        self.assertEqual(["a" * 5], wrapped)

Behaviour Driven Development (BDD) Style

Similar to the AAA pattern, the BDD style uses three other keywords to specify each test case: Given, When and Then. (You can also use And as another keyword.)

Given The Text Wrapper's Width Defined As 10
And Using '-' As Word Connector
When The Wrapper Wrap Text Length is Less Than 10
Then The Text Should Not Be Wrapped

As you can see, "given-when-then" maps to "arrange-act-assert" pretty well. They both simply define a state transition of a Finite State Machine (FSM). You can find more on this in the Uncle Bob's article. Some differences:

  • BDD is more "outside-in", which means that it emphasises more the external behaviour
  • With BDD, you need to define a domain specific language to write your test specifications. Because of this, usually you'll need a different framework. One example for Python is behave.

Golden Rule Of a Unit Test

In general, this is a good rule for each unit test case:

Each unit test case should be very limited in scope.

So that:

  • When the test fails, no debugging is needed to locate the problem.
  • Tests are stable because dependencies are simple.
  • Less duplication, easier to maintain.

Think About It

Which of the following will you rarely see in a unit test case?

  1. if statements
  2. Random numbers
  3. A test without any checks
  4. A test with a lot of checks
  5. for loops
  6. None of above

Why?

Further Information

  • For historical reason, the standard module unittest is not "pythonic" enough. It is not fully compliant with the PEP8 convention. For example, setUp and assertEqual are not good class method names (probably should be setup and assert_equal).
  • nose has many other features. Check the online documentation.
  • There are also many other testing frameworks for Python.
  • One of the choices is py.test. It can also run unittest and nose style tests, and is very easy to use.