Technical Debt

Ward Cunningham posted a video on Technical Debt, and Kent Beck and Ed Yourdan (and I) have been tweeting about it. Let me share some thoughts with you here.

In Ward’s video, he refers to some folks’ notion that is is OK to write the program a little less well with intention to clean it up later. He’s not in favor of that. He says that the program as written should always reflect all of our understanding of the problem and the solution, as of the time we wrote it.

Ward points out that as time passes, our understanding will move beyond the design as reflected in the code. This amounts to “interest due” on the code, and if too much interest comes due, you will be unable to make much real progress, just as when our credit card debt gets too high, we have no money left to actually buy anything.

Meanwhile, Kent and Ed were tweeting about transparency, and Kent gave the example that “many people argue that customers e.g. ‘don’t need to know” about refactoring.'” My purpose here is to talk about when customers do need to know … and when they don’t.

In one sense, the customers always need to know. Something like this:

Hi Customer. Producing stories at a reasonable and essentially constant rate requires that the code be kept in good order. The grubbier the code, the slower we go. Therefore, the professional thing to do is to keep the code in the good range. We do this by continuous refactoring. It is part of our responsibility to you and is part of developing software for you. We promise to do it.

With that understanding in place, we need not track specific time spent refactoring. In fact, we probably shouldn’t even be able to track it: we will ideally move so smoothly and rapidly between refactoring and testing and coding and designing that we can’t keep track of the time.

However, as a program evolves, there’s a good chance that the Design In Code will not include all the good things we now understand. We have a better Design In Head. When the design in our head is enough better than the one in the code, it can pay off to bring the code closer to what we now understand.

One way to do that is to tell the customer — the topic of the Beck/Yourdon/Jeffries tweets.

Hi, Customer. We have some new understanding of good ways to build a program like ours, and if we put some of that in, at a cost of N days, we should be able to speed up our development by Y percent. What do you think?

Note that this conversation really needs to include the cost (N days) and the benefit (Y percent). If we can’t give this information, how can the customer possibly make a good decision?

Even if do we provide this information, what will the customer choose? Well, the customer wants features. Features today, features tomorrow, features every day. Features now very likely seem better than some higher fraction of features at some future time. So the customer is likely to want to defer the refactoring, especially since it isn’t really necessary. So they may say “later”, or they may ask if we could just do a little bit each week.

When a team finds itself wanting to ask permission to do some big refactoring, I see that as a danger signal. Two danger signals, in fact:

  1. We have failed to notice a deviation between what we know and what we have expressed in the code for long enough to let the deviation get large.
  2. The best way we can think of to fix the problem is to get everyone out of the pool and refactor like demons.

Well, if that’s the best we can do, we have a responsibility to open the question with our customer, because there is an important and major monetary decision to be made. Should we leave the road deteriorating and experience continuing slowdowns, or should we stop, resurface the road, and then get traffic going again?

However, that’s not the best we might possibly do. Remember what we said at the beginning?

Producing stories at a reasonable and essentially constant rate requires that the code be kept in good order. The grubbier the code, the slower we go. Therefore, the professional thing to do is to keep the code in the good range. We do this by continuous refactoring. It is part of our responsibility to you and is part of developing software for you. We promise to do it.

OK, then. We see a new design that we wish we had. We refactor the code daily because it is part of our promise. Let’s figure out how to morph the code from where it is, to where it needs to be, in the same way we always do it, incrementally, inch by inch, step by step (slowly I turned). If we can do that, we won’t need to ask permission to clean things up.

Can every refactoring be done inch by inch, step by step (slowly I turned)? Well, yes. Do I know how to do every one? Well, no. What I do know, however, is that by the time I have finished doing one as a blob, I have always then known how I could have done it incrementally. So I conclude that I have some learnin’ to do … but that it is always possible.

So we don’t have to tell?

Sorry. We do have to tell, because we want everyone on the team, including the customer, to know what is going on. We just don’t have to ask permission.

As I worked on Story Abra and Story Kazam, I ran across some of that code that uses the old Bungle object, so I refactored it to use the new Fangle object. When I was done, I searched for remaining uses of Bungle and there are only 31 to go. That’s down from 137. Yay, us!

5 Responses to “Technical Debt”

Sebastian Kuebeck

February 19, 2009

11:46 am

permalink

> Sorry. We do have to tell, because we want everyone on the team,
> including the customer, to know what is going on.
> We just don’t have to ask permission.

I think you are perfectly right!
Alan Cooper says in a different context that you should never ask your user (or customer) to design the application themselves…

http://www.youtube.com/watch?v=sNWBnCazIcU

I think it’s the same with refactoring. It’s the developers job to find out when to do it and why.

Sebastian

mabotelh

February 19, 2009

1:03 pm

permalink

“Can every refactoring be done inch by inch, step by step (slowly I turned)? Well, yes.”

I think that the inch by inch approach work for most of the cases and in my experience has been the best approach.

One case where this can be hard to accomplish is when your “refactoring” is very radical, perhaps in the order of switching from say Java to .NET.

In this case you can be put in a position of having one foot in the boat and one on the shore. Not stopping traffic like you mentioned in your post might mean a quickly rising technical debt.

Could you please elaborate more on this topic?

Mauro

jonkern

February 20, 2009

10:45 pm

permalink

Regarding “…some folks’ notion that it is OK to write the program a little less well with intention to clean it up later… He [Ward] says that the program as written should always reflect all of our understanding of the problem and the solution, as of the time we wrote it.”

How can it not? It certainly cannot reflect more than your understanding.
Or, conversely, how can it ever truly reflect all of your understanding?

I can guess that Ward means “Don’t do something stupid and leave a turd in the program.” Which of course presumes that the developer knows in the first place what constitutes quality design and good coding.

Not sure where these fit in to the continuum that represents Ward’s comment…

1) Many times I have the opportunity to “grow” a feature over time. Though I might have a long-range vision that meets business needs, we have the ability to release a subset of the whole and achieve business value sooner than were we to implement our entire understanding. Of course, some of the understanding involves realizing that we can get value with less. So, because I think this is a smart thing to do, maybe it doesn’t fail ward’s test.

2) Once in a while, I will jam out a feature with some potentially ugly code because that is the best choice to get it out the door. Getting it out the door is a big win for the business/product. This allows me to get value now. But we then immediately come back to this hack to refactor and do it right. This allows us to avoid technical debt in the future.

Never say never.

But guidance of good things to do when given the chance is always a good thing, as you and Ward are trying to do. However, hoping that all developers know how to best balance their efforts to the current understanding is a conundrum.

— jon

Dirk Riehle

February 21, 2009

9:35 pm

permalink

Manuel Klimek pointed me here.

If the financial metaphor is apt, and the customer is like an investor, then you should disclose any relevant liabilities, of which technical debt would be one.

Dirk

mheusser

March 24, 2009

2:48 pm

permalink

I think we use different rhetoric, but this is essentially a sub-selection of my opinion on tech debt. Oh, there are different dynamics and possibilities involved, but, in general, this is solid advice i’d support.

–matt heusser

Recent Articles

XProgNew – Pivot?

Experience with Sinatra, and some thinking, cause us to look at a shift in approach. Martin Fowler was probably right.

Codea Calculator

Based on a simple example on the codea.io forums, I decided to write an article showing all the stuff I might do on a production calculator project. Wow.