Calculating technical debt (from “Executive Engineering”)

There are roughly five things you need to know about each debt obligation in your system:

1. Principal – What would it take to fully pay it off?
2. Interest – How much energy is lost just putting up with it?
3. Increase In Principal – How much bigger will the payoff be in the future?
4. Increase In Interest – How much more energy will we lose in the future?
5. Payoff Events – Are there any inflection points in the future that will necessitate a sudden payment in full?

You may notice I described the debts here in terms of creative energy, not just time. This is deliberate because technical debt costs us cognitive drag, not just time. Creative engineering work does not happen at a constant speed. An engineer might say only 10% of their time is spent on a repetitive task but if you dig deeper it’s appears to be sapping a majority of their emotional and intellectual initiative. Toil like that can push the team to get distracted or snack on low-value work to avoid the more direct, frustrating work. Asking the team how much of their creative energy they feel like they’re losing is a better measure of how much enthusiasm and insight is lost. Which is what we actually pay our engineers for.

Jack Danger, Executive Engineering

One example he gives is a ballooning postgres database which isn’t causing any issues now, but will be out of space in about 18 months.

Principal	Interest	Increase in Principal	Increase in Interest	Payoff Event
3 months * 1 team (to migrate to sharding)	zero	Low (migration takes slightly longer with more data)	zero	18 months

Another good example is an async app on MongoDB which should be a plain old app on a relational DB. Testing the async code required creating and maintaining a testing library, and the document-oriented data model makes everything more complex without providing value. They say that anecdotally, 75% of their energy is spent just fighting the system. But it would take a full year to rebuild.

Principal	Interest	Increase in Principal	Increase in Interest	Payoff Event
1 year * 1 team	75%	High (a function of features * data complexity)	High (a function of engineer count)	None

On comparing these two debts, the book says:

Assuming both systems are roughly equal in their importance to the company, I would advise this director to stem the bleeding on the async MongoDB system by curtailing new development in it.

We can cap the growing principal and interest if we don’t add new features into that system. To do this, we’d need to staff a one-time migration to design a replacement system that new features can go into and write a full proposal on (eventually) migrating all existing features to the new system. (It’s important to write that detailed proposal in order to test whether it’s possible to one day fully decommission the existing system.

Considering the timelines involved, I’d advise this director to give the next year of time to the MongoDB team to build and start to use a new approach, then spend the second year focusing on the sharded Postgresql migration. If the company hasn’t totally transformed after that, the third year the MongoDB team can finish decommissioning the original app.

That’s all. I just liked this method of calculating and reasoning about tech debt and I didn’t want to forget it.