45% with no technical debt

I took stock of the year’s progress on one of the projects that I monitor for technical debt. The 70 developers were under considerable pressure to sidestep writing unit tests for all the lines and branches they added. The team was adding code at a rate 40% higher than last year. Furthermore, Sonar wasn’t fully functioning for the dot Net part for most of the year.

However, at the end of the year, 45% of the developers had no debt. This means that they covered all branches and lines and added no static violations (PMD, CheckStyle and FindBugs). Most also increased the quality of the programs in addition to avoiding debt.

So, although not TDD, at least no technical debt for nearly half even under project pressure. Admittedly, many paid back the debt after code release but at least the debt was paid back. The rest will be paying back the debt early this year.

haven’t turned the corner yet

Well, I noticed that several developers were responding to the technical debt emails they received within a few hours of submitting technical debt to the source code repository. Anecdotal evidence seemed to indicate that the tide might be turning.

I used two indicators to determine if this were the case.

  1. Change in technical debt over time. I checked this week versus last week. No dice. The debt increased by over $10,000 in only one week. (As measured by the basic Sonar technical plugin using the standard settings.)
  2. Change in net contribution. I have a net contribution score for each developer’s code quality contribution. It combines both the good and the bad. So, are the developers net contributors or net debtors? For the most recent month, the number is negative.

So the answer is no. But we haven’t begun developer training yet, only manager and lead training. So stay tuned, the training should be key in turning the tide.

does faster velocity reduce commit quality?

correlation of velocity to good commitsOne assertion I hear, especially from managers, is that when the work volume increases, the commit error rate increases. By commit error, I mean a source code commit that has some form of technical debt, be it an uncovered line/branch, static code violation, duplicated code, tangle or exceeding a complexity metrics threshold.

So I plotted monthly velocity (file commits) on the vertical axis and the corresponding monthly commit success rate on the horizontal axis. For a 2-million line Java over 50 months I plotted a linear regression line on the XY plot.

What were the results? Absolutely no correlation. One the busiest months had the best quality rate and one had the worst.

This jives with my experience. I’ve often seen the busiest teams with the tightest deadlines still deliver debt-free projects. They have a constant focus on their technical debt and strive to commit code with no Eclipse yellow warnings and all green code coverage. On the other hand some teams seem to ignore quality figuring no one seems to care.

when is a program too complex?

Everyone seems to agree that programs shouldn’t be too complex. However, for technical debt purposes a specific complexity limit needs to be set. Sonar sets the limit at 60 McCabe Cyclomatic Complexity points. The Software Engineering Institute sets it at 50. It considers programs larger than that to be virtually untestable.

In my experience complexity correlates to defects. One logistic regression model I built with actually program failure data shows a near 100% chance of failure when complexity exceeds a certain amount. Troster in 1992 and Craddock in 1987 report the same.

Complexity is a coding metric but can also be a proxy for design issues. In fact it’s really the only design metric available in the C# Sonar plugin. So tracking the complexity metric in that environment serves to reduce defects and also to improve design quality. In a Sonar Java environment there are other design metrics like cohesion and coupling that can be used to better target which complex programs should be re-factored.

Unfortunately, complex programs can easily become a large part of the code base if they’re not monitored from the beginning. A large program attracts other code like flies as the inevitable one-line changes are made to the program. The maintainer doesn’t really know what the program does. They just find a safe place to insert the method or line, do a happy path test and declare victory, tip-toeing away from the class before it blows up.

In a new one-year old 1 million line system, only 3% of the classes exceed the 50 cc limit. That doesn’t seem so bad until you realize those 3% represents 34% of the code base by LOC.

It’s hard to remediate that kind of debt. Programmers resist re-factoring a class they don’t understand, especially for a one-line change. But you have to start somewhere. I usually run the cohesion metric, LCOM4, and see if there’s a natural break in the program structure. Eclipse allows methods to be extracted, which helps in the re-factoring. Train developers on safe re-factoring using the good resources out there on re-factoring.

Moral of the story for class complexity? Like voting in Chicago, track it early and often. Re-factor and fix it as you go.

How much does zero technical debt cost?

A question I’m often asked is how much extra does it cost to operate with no debt?  What’s usually meant by this is how much extra effort is required to have 100% unit test branch and line coverage.  There’s the effort involved with managing the extra code that’s involved in unit tests.  About 35% of the commit activity is for unit tests on a 2 million LOC project with 37,000 unit tests.  There’s effort in maintaining the tests and re-factoring the tests or changing them if an API changes.  Sometimes the tests are brittle or change global static causing an unrelated test to mysteriously fail.  So there’s work involved.

Studies at 3 major companies show that the extra effort is around 20%.  This is the amount show by all three, independently of each other.  So that’s the number I use.  It jives with my own experience as well.

Of course, one can always say the the extra testing costs nothing because you recover the testing cost by defect reduction.  True enough but most project managers are responsible to estimate specific tasks and projects which don’t usually include (or track) defect remediation.  They need to estimate code and test effort.  So it’s fair to consider the testing burden.