How to write a good unit test
This post provides some guidelines about how to write a good unit test. There are some examples in Python with unittest.
Introduction
In my last post I wrote about software quality not being taken seriously sometimes.
Tests are a good part of the software quality so I wanted to write a bit about the testing effort we should be doing in our code.
Having said that, let us delve in the unit testing world with some examples in Python!
What is a unit test?
A unit test is a test that checks the immediate dependencies of a module.
For example, given this simple function that makes a HTTP request to an URL:
# api/health.py
import requests
from logging import Logger
def service_is_healthy(service_name: str, logger: Logger) -> bool:
response = requests.get('https://{service_name}.example.com/health')
ok = response.status_code == 200
if ok:
logger.info(f'Service {service_name} is healthy')
else:
logger.error(f'Service {service_name} is not healthy')
return ok
A unit test would be something like the following one:
from unittest import TestCase
from unittest.mock import MagicMock, patch
class TestApiHealth(TestCase):
@patch('api.health.requests.get')
def test(self, mock_requests_get: MagicMock):
service_name = 'my-service'
logger = logging.getLogger(__name__)
mock_requests_get.return_value = MagicMock(status_code=200)
healthy = service_is_healthy(service_name=service_name, logger=logger)
self.assertTrue(healthy)
The immediate dependency (requests) is replaced by a mock dependency that we can modify to change the behavior of our tested function.
What is a good unit test?
It seems simple, we just check:
- What are we calling.
- How are we calling it.
It as simple as checking the immediate lower layer from the piece of code we want to test.
Test name
Naming is the hardest problem in Computer Science Software Engineering, so we need
to provide a name that is meaningful and it actually helps the engineers to identify what
it checks.
A good candidate for our example is test_service_is_healthy_status_ok
.
Mock the dependencies
Unless we are talking about a Singleton that is extremely hard to mock, all dependencies must be mocked.
Thus, it is easy to know the dependencies just by looking at the patch decorators:
@patch('api.health.requests.get')
def test_service_is_healthy_status_ok(self, mock_requests_get: MagicMock, mock_logger: MagicMock):
...
Exhaustive
In this context what I mean by exhaustive is that all the dependencies must have their contracts checked. By contracts I mean the API calls.
So, if you have two mocks in your test, you would need to add assertions for each mock that check not only the number of calls but the arguments of each call. If you do this that way you are ensuring the contract between your module and the lower-level layer.
@patch('api.health.requests.get')
@patch('api.health.Logger')
def test_service_is_healthy_status_ok(self, mock_requests_get: MagicMock, mock_logger: MagicMock):
service_name = 'my-service'
mock_requests_get.return_value = MagicMock(status_code=200)
healthy = service_is_healthy(service_name=service_name)
self.assertTrue(healthy)
self.assertEqual([call(f'https://{service_name}.example.com/health')], mock_requests_get.call_args_list)
self.assertEqual([call(f'Service {service_name} is healthy'), mock_logger.info.call_args_list])
Minimal
A test must check the least assertions as possible. I am not talking about having tests for each of the assertions, but for the group of assertions.
For example, if we intend to test if logging messages are written, do not create a test for each span expectation, but one test for all of them.
@patch('api.health.requests.get')
@patch('api.health.Logger')
def test_logging_when_service_is_healthy_status_ok(self, mock_requests_get: MagicMock, mock_logger: MagicMock):
service_name = 'my-service'
mock_requests_get.return_value = MagicMock(status_code=200)
healthy = service_is_healthy(service_name=service_name)
self.assertTrue(healthy)
self.assertEqual([call(f'Service {service_name} is healthy')], mock_logger.info.call_args_list])
@patch('api.health.requests.get')
@patch('api.health.Logger')
def test_logging_when_service_is_healthy_status_internal_server_error(self, mock_requests_get: MagicMock, mock_logger: MagicMock):
service_name = 'my-service'
mock_requests_get.return_value = MagicMock(status_code=500)
healthy = service_is_healthy(service_name=service_name)
self.assertFalse(healthy)
self.assertEqual([call(f'Service {service_name} is not healthy')], mock_logger.error.call_args_list])
Why do we need to leave the assertTrue
or assertFalse
? We need a hook to ensure the function is running as expected, and as leaving
that check is not much work, I always recommend ensuring that the behavior is the expected one in all tests.
Check what not happens
It is important to check what happens but also what does not happen.
Coming back to our example, ensuring that the error log is not written at all when everything works perfectly is paramount:
@patch('api.health.requests.get')
@patch('api.health.Logger')
def test_logging_when_service_is_healthy_status_ok(self, mock_requests_get: MagicMock, mock_logger: MagicMock):
service_name = 'my-service'
healthy = service_is_healthy(service_name=service_name)
self.assertTrue(healthy)
self.assertEqual([call(f'Service {service_name} is healthy')], mock_logger.info.call_args_list])
self.assertEqual([], mock_logger.error.call_args_list])
External dependencies
This is a personal preference but I do not like having to install packages for testing my projects.
This is why I would rather use unittest than pytest in Python. I know there are others that do not think like me, but having another package whose upgrades, deprecations and removals need to be taking care of is not worth it for me.
Other languages like Ruby where there is some consensus in the community to use another one (rspec). I think I have never used Test::Unit in all the years I have worked with Ruby.
I recommend using the most popular and (if possible built-in) testing package of your programming language or framework you are basing your project on.
Conclusion
Hope this short article has given you some pointers about how to create a good and useful unit test.