Today there is high focus on great quality on production code. With production code I mean code that is part of the application. Test code tests production code. I see little worry about quality of test code, only production code. Test code can come in any form or shape, just as long production code has great quality. I find this dangerous.
Let’s talk about numbers
We have many types of tests and many level of tests. In order to write a few lines of production code, many lines of test code are written. Once, while working on a feature, I kept track of the number of lines I changed or created for tests and for production purpose. Test code represented > 80%.
Unit tests, integration tests, user acceptance tests, … represented 4 times more (at least). All this code needs to be maintained. Every time production code behavior is changed, tests will brake (or else tests are not testing nothing). Test code will have to be changed.
Let’s talk about facts
Test code is used to:
- make sure future changes don’t break existing behavior
- help write production code (if TDD is used)
Many developers only focus on the first one, regression testing. The most forgotten aspect is documentation. I hear “tests are documentation” often, but “walk the talk” doesn’t happen.
The first point has great consequences. Regression testing requires already existing tests to be maintained. There is no tests for tests. When test code is changed, it is hard to know if any test stop testing what it was written to for.
Test code is code
Production code is intended to implement behavior. Test code tests that behavior. Test code can be slower, consume more memory, be less efficient, but it is code anyway. Many books address refactoring, design patterns, best practices, clean and simple code. All of these can be are applicable to test code. So please do.
We are in a age of refactoring. Refactoring to make the code readable. Refactor to get rid of legacy code. Refactoring is the “art” of changing code without changing its behavior. Refactor test code, violently. Refactor it to make it simpler, readable. Most of all, refactor test code to make it maintainable.
Software systems become hard to change when its code is hard to change. Production code and test code. If changing a line of code, dozens of tests get broken, it means it will take time to fix those tests.
Test code can also make it harder to change software systems and maintain then, as test code represents most of your code (or it should). Production code maintainability is seen as the reason for a system to enter a Legacy state. This point of view will change with time.
Tests are documentation
One tendency I see is for developers to jump first into production code for answers. Hardly ever see them going for test code. So much talk on how write good tests, but it is not being used for one of its purpose. I believe going for production code to understand it, beside force of habit, it is a symptom, a smell. A smell that your tests are not readable.
The above code exemplify what is common in tests today. Tests are too verbose. I was nice on the assertion section, usually it’s worst. So many books describe how to produce great code. All is forgotten when it comes to tests.
Apply the same effort in writing good code while writing tests. The same principles apply. But test code should be writing with the purposes of testing and documenting. The ideal scenario is to have 3 lines per test (any type of test):
- fixture or given
- execution or when
- assertion or then
From those 3 lines we should understand what that test is doing. Extract code into small methods, use helper classes, inheritance, what ever it takes, to make the test the simplest to read, understand and maintain.
The test code is not only 3 lines, but the test method is. To see how BookRepository is setup to return the “xUnit Test Patterns”, it can be drilled done, but it is not clotting up the test method.
Test coverage is easy to measure. We can even automate it on our CI server. Much attention is given to Unit test coverage. Little attention is given to the coverage of other types of tests. Even less attention is given to tests quality and redundant tests.
Is a code base with 100% test coverage of great quality? It depends. Quantity is not a synonym of quality. In certain situations, a simple integration/component test can test what 10-20 unit tests never will. Maintaining 1 test is easier than maintaining 10-20 tests.
Writing tests requires tact. It is always about writing the right amount of tests, the type of right, communication and simplicity. I like “killing two birds with one stone” and use implied testing. Imagine the following Java code:
For this example code I would write 1 unit test. The test below covers 100% of the above code and tests every behavior. Much is tested implicitly.
Don’t ever forget maintaining 1 test is always easier than maintaining 2 or more. Be smart.
Tests are used for regression purpose. As a system progress and changes, tests have to maintained and changed. Tests are not covered by tests, having a higher risk of making tests useless without knowing. Worthless tests are toxic, it give us a false sense of security: if you have 20 tests testing a piece of code, you feel confident you can change it at will.
Coverage might be great and proudly yield 100% (unit) test coverage. How good are they? How hard is it to change them? Could there be less and simpler tests and the same coverage? How easy is it to read them?
Treat test code almost as if it is production code. It is code just the same! Apply all the techniques you would apply to production code. Do not let test code become hard to change. It will make the system hard to change.
Make test code simple, readable and maintainable. Take great care on its design. Ideally test code should describe the problem better than production code itself.