Python unittest Coverage
Summary: in this tutorial, you’ll learn how to use the Python unittest coverage command to generate a test coverage report.
What is test coverage?
Test coverage is a ratio between the number of lines executed by at least one test case and the total number of lines of the code base:
test coverage = lines of code executed / total number of lines
The test coverage is also known as code coverage.
The test coverage is often used to assess the quality of a test suite. If the test coverage is low e.g., 5%, it is an indicator that you’re not testing enough.
However, the reverse may not be true. For example, 100% test coverage is not a guarantee that you have a good test suite. In other words, a test suite with high coverage can still be of poor quality.
Unittest coverage example
We’ll use the following project structure to demo the unittest coverage. Note that you can get the source code from this tutorial.
D:\python-unit-testing
├── shapes
| ├── circle.py
| ├── shape.py
| └── square.py
└── test
├── test_circle.py
├── test_square.py
└── __init__.py
To generate a coverage report, you need to carry out two steps:
First, run the coverage module to generate the coverage data:
python -m coverage run -m unittest
Second, turn the coverage data into a report:
python -m coverage report
Output:
Name Stmts Miss Cover
-----------------------------------------
shapes\circle.py 9 0 100%
shapes\shape.py 4 0 100%
shapes\square.py 9 0 100%
test\__init__.py 0 0 100%
test\test_circle.py 14 0 100%
test\test_square.py 14 0 100%
-----------------------------------------
TOTAL 50 0 100%Code language: plaintext (plaintext)
To generate the coverage report in HTML format, you change the option of the coverage module to HTML like this:
python -m coverage html
Output:
Wrote HTML report to htmlcov\index.htmlCode language: CSS (css)
The output indicates the location of the HTML coverage report htmlcov\index.html under the project folder.
If you open the index.html file of the htmlcov folder, it’ll look like the following:
Examining unittest coverage detailed report
First, add the perimeter() method to the Circle class as follows:
import math
from .shape import Shapedef __init__(self, radius: float) -> None:
if radius < 0:
raise ValueError(‘The radius cannot be negative’)
return math.pi * math.pow(self._radius, 2)
return 2 * math.pi * self._radius
Code language: Python (python)
Next, gather the coverage data by running the following command:
python -m coverage run -m unittest
Then, generate the coverage report by executing the following command:
python -m coverage report
Output:
Name Stmts Miss Cover
-----------------------------------------
shapes\circle.py 11 1 91%
shapes\shape.py 4 0 100%
shapes\square.py 9 0 100%
test\__init__.py 0 0 100%
test\test_circle.py 14 0 100%
test\test_square.py 14 0 100%
-----------------------------------------
TOTAL 52 1 98%Code language: plaintext (plaintext)
The coverage now is 98% in total and 91% in the shape\circle.py module. This is because the perimeter() method is not tested.
After that, generate the coverage report in HTML format:
python -m coverage html
The circle.py has 11 statements. The test executes 10 of them and misses one statement. Therefore, the test coverage is 10/11 ~ 91%.
Finally, click the circle.py module for the detailed report:
Summary
- Use the
python -m coverage run -m unittestcommand to gather coverage data and thepython -m coverage reportcommand to generate a coverage report. - Use the
python -m coverage htmlto generate the test coverage report in HTML format.