I did something today that I haven’t done in a long time: I wrote an automated unit test. I loathe writing unit tests, because despite the rhetoric, I have yet to experience a dramatic increase in either velocity or code quality. Unit testing, like a sub prime mortgage, promises great returns with minimal risk and costs. The reality is very different.
Subprime mortgages were sold as an investment that offered greater returns with just a minimal increase in risk. They turned out to be toxic and lead directly to a global financial meltdown. Thankfully, your unit tests will not have this far reaching impact. The risks and drawbacks will be localized, but they do exist.
Unit testing is an investment and evaluated with an eye on ROI. There are fixed costs such as build technologies and environments as well as continuing costs with the 1.5 to 2x increase in development time for a feature. The promised return is an assurance of code quality. Similar to mortgages, the core assumption is the market is stable.
The market is never stable. When the market shift happens to your group, you will be left with a suite of tests that validate a failed idea. The group now has to take the time to validate and decommission the old tests and mocks along with writing new tests or face losing the stellar code coverage it attempted to create. The tests have no future return than to prove what the developer already knew; the code has changed.
Another risk from unit testing is narrowing the developer’s scope to a single horizontal slice. The most impactful production issues arise when the interdependent sections of code fail to act as expected. In the financial world, failure to see the big picture means worldwide depression. In software, it just means missed deadlines or financial meltdowns.
A unit test can tell you the conditions under which a function returns an NULL or an empty List. It call tell you what occurs when a specific caller receives a NULL or an empty list. But it cannot tell you how every layer, including the UI, will react when the first function always returns an empty list, instead of a NULL. This situation is especially true when mocks or another system for representing idealized behavior is used.
A global view is critical in understand what each piece of the software is supposed to do and can only be gained by using the system as the user would. At the time of writing, humans are the only ones that can tell if software will work for humans.
Treating automated testing as an investment, instead of a mantra or mythodology, reveals the areas where it does have value. For example, the return is very quick when it is used to recreate complicated issues or validate situations where multiple inputs translate to a few well known outputs. Unit testing has value, but that value must always be measured against the costs.
Did you like this content?
Related Posts:
Fast, Cheap, and Good; How to get all three.
High-quality product can be created quickly and cheaply if the available resources are utilized wisely.
On it becoming Day 2 and Leaving Amazon
Making the jump during a pandemic was daunting but there is just something about startups.
Maker's Space, Manager's Space
Manager vs Maker space is just as important as Manager vs Maker time
