Unit Testing Clarified, Part 1

This article is one of the “IT Terminology Clarified” series.

In software testing, there are many terminologies that are used wrongly by IT professionals. In this article, I will talk about ‘Unit Testing’, which is misused a lot in software projects.

Unit testing is a type of white box testing performed by programmers at the source code level. Software testers (QA team) should have no concerns about it. A more correct term for ‘Unit Testing’ would be “Programmer Test” (I didn’t invent this term, I read it from a book. Pitifully, I forgot the name of that book).

Last week, an IT manager showed me their release document, which has “QA unit testing” (by manual testers). I pointed out the error. This manager was quite open-minded and corrected it immediately: “QA functional testing”. I have seen this mistake many times; however, not everyone was willing to admit that. Though I don’t remember anyone argue on this (the mistake was so obvious), a common response from the tech leads: “put aside the differences of the terminology, blah blah blah, …”. Frankly, I have lost interest in listening to a tech lead who does not even understand this basic stuff. It is pointless to discuss Test Automation, CI/CD or Continuous Testing with him.

What is a Unit Test?

A software product that passes comprehensive programmer tests (i.e. unit tests) can still fail on many functional tests. That’s because unit tests are from a programmer’s perspective, and functional tests are from a user’s perspective. A unit test is a type of automated test. You must have heard of the xUnit framework. JUnit, created by Kent Beck and Erich Gamma, kickstarted the unit testing revolution in 1998.

“Never in the field of software development have so many owed so much to so few lines of code” — Martin Fowler, on “xUnit Test Patterns” book.

However, many programmers do unit tests poorly. Michael Feathers, the author of ‘Working Effectively with Legacy Code”, wrote a famous post “A Set of Unit Testing Rules” in 2005:

A test is not a unit test if:

• It talks to the database
• It communicates across the network
• It touches the file system
• It can’t run at the same time as any of your other unit tests
• You have to do special things to your environment (such as editing config files) to run it.

Some programmers would think: “How could that be possible?” That was how I thought when I first saw this post. Luckily, I had an opportunity to pair-program with a world-class programmer later. On the first day, he pointed out that my unit tests were not unit tests at all, “the tests you wrote restarted EJB bean. They are not unit tests”. Then he wrote some real unit tests that totally blew me away. Michael Feather’s unit testing rules are correct. For example, for unit testing DAO (data access object) classes, you can use in-memory database.

Differences between a unit test and a functional test

I know the audience of my blog is mostly interested in functional testing. There are three clear factors that distinguish unit tests from functional tests:

1. Who does it? - Programmers, and only programmers, write unit tests. That’s why unit tests are also known as ‘programmer tests’.
2. Scope - Individual function/method in source code.
3. How long does it take to run? - To help you understand the difference, I come up with a 30X factor rule (not based on science, but rather a guide for myself when working on these three kinds of automated tests).

A programmer shall always run and pass all the unit tests before he checks in the code. It is possible as the execution of good unit tests is fast. I remember that in 2006, my team created ~1800 JUnit tests (mostly done by me) which took about 6 minutes to run. There was room for improvement ( Computers at that time were quite slow).

--

In Part 2, I will answer some questions on Unit Testing.

This article was originally published on my Medium blog on 2021-08-22.

Further reading:

• Testing Clarified series