Unit Testing is Software's Subprime Mortgage

Unit testing, like a sub-prime mortgage, promises great returns with minimal risk and costs.

Posted by Tejus Parikh on November 17, 2013

Bad debt can be a noose around one's neck

I did something today that I haven't done in a long time: I wrote an automated unit test. I loathe writing unit tests, because despite the rhetoric, I have yet to experience a dramatic increase in either velocity or code quality. Unit testing, like a sub prime mortgage, promises great returns with minimal risk and costs. The reality is very different.

Subprime mortgages were sold as an investment that offered greater returns with just a minimal increase in risk. They turned out to be toxic and lead directly to a global financial meltdown. Thankfully, your unit tests will not have this far reaching impact. The risks and drawbacks will be localized, but they do exist.

Unit testing is an investment and evaluated with an eye on ROI. There are fixed costs such as build technologies and environments as well as continuing costs with the 1.5 to 2x increase in development time for a feature. The promised return is an assurance of code quality. Similar to mortgages, the core assumption is the market is stable.

The market is never stable. When the market shift happens to your group, you will be left with a suite of tests that validate a failed idea. The group now has to take the time to validate and decommission the old tests and mocks along with writing new tests or face losing the stellar code coverage it attempted to create. The tests have no future return than to prove what the developer already knew; the code has changed.

Another risk from unit testing is narrowing the developer's scope to a single horizontal slice. The most impactful production issues arise when the interdependent sections of code fail to act as expected. In the financial world, failure to see the big picture means worldwide depression. In software, it just means missed deadlines or financial meltdowns.

A unit test can tell you the conditions under which a function returns an NULL or an empty List. It call tell you what occurs when a specific caller receives a NULL or an empty list. But it cannot tell you how every layer, including the UI, will react when the first function always returns an empty list, instead of a NULL. This situation is especially true when mocks or another system for representing idealized behavior is used.

A global view is critical in understand what each piece of the software is supposed to do and can only be gained by using the system as the user would. At the time of writing, humans are the only ones that can tell if software will work for humans.

Treating automated testing as an investment, instead of a mantra or mythodology, reveals the areas where it does have value. For example, the return is very quick when it is used to recreate complicated issues or validate situations where multiple inputs translate to a few well known outputs. Unit testing has value, but that value must always be measured against the costs.

[Photo credit ~ Pil ~, original photo]

Related Posts:

Tejus Parikh

Tejus is an software developer, now working at large companies. Find out when I write new posts on twitter, via RSS or subscribe to the newsletter: