In my previous article about the 4000er summits I wrote a Python script to scrap table data from a Wikipedia article. To finish my original task, I started off with a script for that table. During the development I figured out that I need to expand the script a bit to filter cell content, handle tables with row spans in the column headers and so on. The script should be flexible.

However, tables can be trickier than I thought. There are colspans and rowspans not only in the table header but also in the body somewhere. Also (which I didn't notice) the table with the montains of the alps suddenly had fewer columns in lower rows. Also, some table contain the "header" columns not as cells in the first or second row, but in the first left column. With the current script the json objects would quite odd using weird property names that make no sense at all.

Conclusion: the script wiki2table.py needs to be extended with additional functionality. But when extending a script, one must be sure that still current functionality is preserved and does not break. To ensure this we can use unit tests.

Unittest in Python

Python has a few built in modules as well as external modules to support unit testing. There's a good introduction at Real Python (needs login) that mentions 3 ways in order to setup unit testing. Among these is the unittest package which is included in the Python standard package. This module is well documented in the official Python 3 docs unittest - Unit testing framework. I basically used the information from that documentation. If you are familiar with another language and unit testing there, all this should look quite familiar. Even the assert methods have nearly the same names and parameters.

Testing a standalone application

Most tutorials and information that I found about testing assume that you have written some module and now want to add test to it. There is usually a folder containting the source code in src and the test are located in another folder tests at the same level. The relation between test and the code to test is quite fix and widely used.

In my case, I have a single script that does the cli argument handling and contains some classes that do the actual work. The tests basically could reside somehwere else, so that I can have a structure that fits my website. The test classes and other test data must be located at a different locations so that I can easily exclude these files when deploying my page to the website.

My files that can be downloaded in various articles are located in <project>/source/assets/files/. This folder is part of the content that is deployed during a publish to the website. My test folder is located at <project>/tests/wiki2table where I store all files to test the wiki2table.py script.

Testing the CLI script

The CLI script would be called in the same was as if it was called manually. python3 wiki2table.py --help show a list of suppored command. The test script inherits a class from unittest.Testcase that contains the tests.

My cli_test.py file looks like the following.

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-

import unittest
import subprocess
import os

class TestCli(unittest.TestCase):

    def runCommandAndGetResult(self, args={}, article=''):
        # Get the path of the current file and create the path to the wiki2table.py script.
        pwd = os.path.dirname(os.path.realpath(__file__))
        cli = ['python3', '{0}/../../source/assets/files/wiki2table.py'.format(pwd), article]
        for k,v in args.items():
            cli.append(k)
            if (v != ''):
                cli.append(v)
        result = subprocess.run(cli, capture_output=True, text=True)
        output = result.stdout.strip()
        return output
        #return os.popen(cli).read()

    def test_filtered_csv(self):
        args = {
            '--table': '1',
            '--cols': '1,3,4',
            '--format': 'csv',
            '--noquotes': '',
            '--tag': 'a',
            '--rmtag': 'sup',
            '--tagtext': 'span',
        }
        csv = self.runCommandAndGetResult(args, 'List_of_mountains_of_the_Alps_over_4000_metres')
        data = csv.split('\n')
        self.assertEqual(data[0], 'Nr.;Summit;Height (m);')
        self.assertEqual(data[8], '8;<a href="/wiki/Lyskamm" title="Lyskamm">Lyskamm</a> (Eastern Summit);4,532;')
        self.assertEqual(len(data), 83)

if __name__ == '__main__':
    unittest.main()

The test itself is inside the test_filtered_csv() method. Here we define the arguments for the script fetch the result and then check whether the result contains the expected output by asserting a few assumptions to be fullfilled.

The method runCommandAndGetResult() determines the correct location of the script to call. As I explained before, the structure is fixed where my test file and real script is located. Based on the test file I know where to look for the script. Attached to the script name itself the arguments are build, a subprocess is launched and the result that's written to standard out is fetched in the result. This is a string, stripped by the last newline and then returned to the caller.

The script should be extended so that all command line arguments are used to see if these are interpreted correctly. The snippet above is missing a few.

Testing the classes

The functional test of single classes or class methods are done by testing the classes directly without using the subprocess to call the cli script.

Here we need to import the file and the classes, to be able to instanciate them in a test method. Again, starting from the location of the test script we import the cli script.

The table_test.py looks like this:

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-

import unittest
import unittest.mock as mock
import os
import sys

sys.path.append(os.path.dirname(os.path.realpath(__file__)) + '/../../source/assets/files')
from wiki2table import Wiki2Table

class TestTable(unittest.TestCase):

    def setUp(self) -> None:
        # Load the html file that contains the wiki article
        pwd = os.path.dirname(os.path.realpath(__file__))
        self.wiki = Wiki2Table('summits.html')
        with open('{0}/assets/summits.html'.format(pwd), encoding="utf-8") as f:
            html = f.read()
        # Create a mock function to return the html content from the file instead
        # of fetching the article from wikipedia.
        mockGetHtml = mock.Mock()
        mockGetHtml.return_value = bytes(html, 'utf-8')
        # Replace the original Wiki2Table.getHtml method with the mock function.
        self.wiki.getHtml = mockGetHtml    

    def test_table_not_found(self):
        self.wiki.getTable(4)
        self.assertEqual(self.wiki.getError(), 'Table 4 not found in summits.html')
        self.assertTrue(self.wiki.hasError())

        self.wiki.error = '' # Reset error to get table 2
        self.wiki.getTable(2)
        self.assertEqual(self.wiki.getError(), '')
        self.assertFalse(self.wiki.hasError())

    def test_fetch_cols(self):
        table = self.wiki.getTable(1)
        self.assertEqual(
            table.fetchCols(),
            ['Nr.', 'Image', 'Summit', 'Height (m)', 'Range', 'Country',
             'Isolation[6] (km)', 'Prominence[6] (m)', 'First ascent[7]',
             'Easiest (normal) route to summit', 'Observations'])

        # Reset columns to fetch to select only some columns
        self.wiki.cols = None
        table.setColToFetch(1).setColToFetch(3).setColToFetch(4)
        self.assertEqual(table.fetchCols(), ['Nr.', 'Summit', 'Height (m)'])


if __name__ == '__main__':
    unittest.main()

Again, this script just tests a few methods, it must be extended to cover all functionality of the Wiki2Table class and the Table class. The formatters would be tested in a separate file.

The setUp() method is included in unittest and can be used to setup some preconditions before running the test. What is done here is, instead of fetching the article from Wikipedia every time, I use a static version that I have downloaded before and stored in a assets file. This is recomended because the Wikipedia pages are always subject to change, so the test might break just because the source is different. Also we do not do too many unneccessary requests to the wikipedia.

Because the wiki article is read from a file and not fetched from the web, we must mock the getHtml() method. Therefore, I create a mock object, set a return value (the html from the assets file) and assign this mock object to replace the original Wiki2Table.getHtml() method.

At the top of the script we extend the path, so that the import of the cli script file works. The import looks the same as if I would import a class from some module.

Run the tests

The test can be run from the command line by calling:

python3 <project>/tests/wiki2table/table_test.py
python3 <project>/tests/wiki2table/cli_test.py

Inside the test directory the test may also called this way:

python3 -m table_test
python3 -m cli_test

To filter a test (e.g. run only the test_table_not_found test) you may do the following:

python3 -m table_test TestTable.test_table_not_found

To run all tests in the test directory:

python3 -m unittest discover -p '*_test.py'

If the current directory is not the test directory, it may by specified with the -s argument and test files are searched staring from this directory.

The output tells how many test have been executed and also reports any error if an assert failed. If everything went well and not test failed, the output looks something like this:

...
----------------------------------------------------------------------
Ran 3 tests in 2.303s

OK

Test cases should be clustered in a neaningful order. I personally would create separate test files for the different classes or components that should be tested. In this example I put the cli tests in one test case and the functional tests of the Wiki2Table class into another one.

The tests need to be extended to cover all cli arguments, all extraction options that the Wiki2Table and Table class offer. In addition the formatter classes need to be tested as well. The examples above just touch some aspects of the script and are far from being complete.