It’s well known that developer quality is critical to successful implementation of Agile software development. Unfortunately, the Scrum Alliance has paid little attention to this concern, not because they think it isn’t important, but because Scrum is designed to be a sort of container for the development process. As I’ve mentioned here before, Jeff Sutherland, Scrum’s co-creator, says explicitly that high performance Scrum teams need XP-style technical practices. Bottom line, no one is arguing against technical practice, and the Scrum movement has by choice not made a lot of noise about it.

That’s going to change. I’ll be writing here from my informed belief that the Scrum Alliance is sincerely interested in people’s success, and that they are doing their best to help people succeed. I say “informed” because I know members of the Alliance leadership and many members of the Scrum training community, and I assess them as good people who are doing their best to help.

As I say, the Scrum Alliance focus on developer quality is going to change. In fact, it has been changing for quite a while. You’ll recall that Chet, Uncle Bob, Brian Marick, Jim Shore and I had a meeting in Chicago to talk about developer certification. That meeting was funded by the Scrum Alliance, with no strings attached. The Scrum Alliance also paid for the followup meeting in Ann Arbor, reported here under Developer Certification and here under Agile Skills Workshop. The result of that was the Agile Skills Project. As I’ve described before, this project is building an inventory of the many skills that developers need in order to be effective in an Agile situation. There are mailing lists on this subject, and a wiki, and things are proceeding well. Here’s a link to the wiki.

One outcome of the Agile Skills Project will be that any piece of courseware, any book, any information offering, can be compared with the Skills Inventory. This will give a quick view of what that course addresses, how deeply it does so, and what it misses. We believe that this will help people better assess what courses to consider.

Now, again, all this has been supported by the Scrum Alliance (and no other organization so far), with no strings attached.

There’s more to come. What I’ll report next is early information, and is subject to change in details, but I expect not in focus.

Scrum Alliance Recognizes Need

The Scrum Alliance continues to recognize the need for developer quality, just as Uncle Bob does, as you do, and as I do here in the corporate towers of XProgramming.com. The Alliance also knows that the notion of “Certified” as they presently use it is weak with respect to any guarantee of quality (there is none) and that it is strong as a marketing approach.

Let me speak to that for a moment. You’re probably aware that Chet and I teach a Certified Scrum Developer course. We have even taught a couple under the Object Mentor flag, by the way, and it was at Object Mentor’s suggestion and support that I became a CST a year ago. Yes, Uncle Bob’s company.

We have had a clear indication that our course that offers the CSM is easier to sell, and easier to fill seats, than a course we would teach without that little certificate. We are clear, always, about what the certificate means and what it doesn’t, and our customers are quite aware of that truth already.

And still, it sells better. We do our best to sell and teach without deceit, and as an insider I am certain that the Scrum Alliance and the Scrum Trainers that I know do the same. There’s certainly a conflict of interest that I feel: on the one hand, I’m the same delightful guy with or without the certificate. On the other, I get more students, and thus more income, when I offer the CSM. On the gripping hand, I also get to influence more people.

What I mostly care about is influence. But I do care about money and I recognize that it’s acting as a force on me. I do my best to be fair about it. This is the problem that any honest marketer faces.

The Scrum Alliance knows that the CSM is an entry-level rating, as is the CSPO. They want to adjust people’s understanding to focus on the CSP (now Certified Scrum Practitioner, possibly to be renamed Certified Scrum Professional.) The CSP is much harder to get: you have to actually work in a Scrum environment, and you have to write an acceptable report convincing people that you have a clue. It’s still not Rocket Surgery but it is definitely a higher bar.

From my connections with the Alliance, I’m sure they are sincere about doing this. As time goes on, we’ll all see what they do. But what about developers and their skills?

The SA Plan for Developers

There is a plan under way to have a new rating, which will probably be called Certified Scrum Developer . (Yes, I’d rather it was called Rank Apprentice Agile Developer but that name isn’t likely. Let’s focus on what it will be, not just on the name.)

The CSD program will focus originally on actual software developers, that is, programmers. (Other specialties will probably follow.) It will require that a developer complete training including Scrum basics (or a CSM), plus an approved course about development. The course must be taught by a “SA-REP”, Scrum Alliance Registered Education Provider. And the course itself must be examined and approved by a representative of the Scrum Alliance who has a clue about what such courses will be. I happen to know who’s doing that for them now, and I’m happy to say that I trust him.

The approval will be based on the course introducing a core of essential ideas and practices, and on offering actual experience doing the practices. It will need to be a programming course, not a sit down and listen course.

At the end of the course, the student will have heard the words we say, will have had experience doing the practices in a lab environment, and will have been told in no uncertain terms that this is their next step on a road to learning that will never end.

Chet and I will be producing a CSD course. I hope that you know us well enough to be confident that it will be darn good. It’ll still be just a few days, but it will be packed with everything we can pack into it.

The SA plans to invite others to produce and offer courses under the SA-REP program. I’d expect that Uncle Bob and his organization would be a very valuable resource for such programs, as would many of our other colleagues in the XP community.

Bottom Line: Perfect, no. Better, yes.

I am confident that the Scrum Alliance sees the need for developer improvement, and that they are working toward making their members aware of the need. I am confident that they are working to provide resources that Scrum teams can use to begin to build the skills that they need. And I’m dedicated to influencing them in the right direction, and to bringing as many people into the situation to help accomplish that.

In the end, what I care about is software development, as narrow and geeky as that might be. I care about other people finding the joy in the craft that I’ve found, and that means they have to discover the joy of life-long learning. I think this Scrum Alliance effort can help with that, and I think that “certification” has little or nothing to do with it. What counts will be what we tell the people who show up.

I welcome your thoughts on this and will leave the comments open at least for now.

And yet, it is risk, not knowing the outcome, working against the odds, putting forth effort, and accomplishing things that express mindfulness, effort and skill, that make life and the things we make most meaningful. And so, what this boils down to is that at least some good measure of what we do should be done the hard way. Choose your tools wisely, that they allow for the development of your skill.

In “skill and toolery”, he says:

These simple charts may help to explain why so many wood workers like to work with hand tools. Skill and attention are matters that create feelings of self-worth and accomplishment. They are the soft side of harsh reality, and the human component that compels engagement in physical and cultural reality.

These things speak to my heart. If I’m good at a few things, it is because of the joy that the craft of making them has brought to me. This applies to all the things I may do moderately well.

If I express software development ideas nicely once in a while, it is as a result of having expressed them hundreds of times in different ways. Some of those ways have worked well, some incredibly poorly. (I apologize to those of you who have had to read so many of them, and offer thanks for your help in the occasional success.)

If I have the ability to look at most any design, most any code, and see how to make it better, it is because I’ve looked at so many bad designs, so much poor code, that needed improvement, and have worked with them to try to make them better. I hasten to add that most of these unsavory collations were my own.

A small lesson for me …

Last week, Chet and I taught a TDD/Refactoring class. We also taught two Agile Introductions, a CSM, a Story Workshop, and a Partridge in a Pear Tree, but the TDD/Refactoring one is on my mind just now. In that class, we demonstrated how we do TDD/Refactoring.

Naturally, duplication was one of the first smells we encountered, and removing it was one of the first refactorings we applied. We did that with Extract Method, and we used the built-in tool. As I think about it now, we’d probably do well to do the first few refactorings by hand. There are at least these reasons:

1. First, the tools work instantaneously and as if by magic, and the audience almost certainly cannot follow what happens as code disappears from one place, replaced by something different, and then appears somewhere else. I know that when I watch coding examples, those quick transforms escape me, and I generally know what to look for.
2. Second, what’s important about this refactoring, and any refactoring, is what’s going on in our mind … and that is best learned by looking at what we do with our hands.  We perceive the code transformation better if we move the code around ourselves. We learn it better.
3. Third, real-world refactorings can be aided by tools but the tools won’t do them, they’ll just do the steps. We learn to proceed in steps, not by pushing the magic button, but by proceeding in steps.

Henceforth, when we do these demonstrations, I’ll make a point of doing things by hand for a while. The magic tools, as Doug Stowe suggests, may decrease risk, but they also decrease our skill.

… and a question for us all

Which of our tools are increasing our skill, and which ones are standing in the way? Which of our tools are bringing joy to our lives, and which ones are standing in our way?

Jim Shore has written a short item on the above subject. I’ll begin by paraphrasing him, following his outline closely.

In a nutshell, he’s saying that customer acceptance testing using Fit (and FitNesse I imagine) isn’t paying off for him. His objection seems primarily to be that the “natural language” tools don’t pay off.

Two things drive this. First, the people in the customer role don’t want to take the time to write the examples. However, they don’t trust examples written by others. The result is that the responsibility gets handed off to testers, which defeats the purpose.

Second, the tests that Jim’s teams get with Fit seem to be end-to-end, and therefore slow, and brittle.

Therefore, Jim is no longer recommending this kind of testing. Instead, he relies on close communication with the on-site customers (product owners), only sometimes creating the examples that correspond to conventional Fit tests, and then turning those examples into TDD tests.

In a couple of tweets, Jim also tells me that his teams do create regression tests, and of course they do exploratory testing as any wise team would. When exploration finds things, or when defects are reported, then the team realizes they are too confident, improves their practice a bit, bears down a bit, and gets back on track.

On the one hand, I find Fit and FitNesse to be an intense pain to use, and in my gut I suspect that if I were made to use it, I’d refuse. Or, I hope, come up with a better idea.

On the other hand, I believe that examples are the best way to communicate about any complex requirement, and I’m concerned that Jim’s teams may not be working on the complex end of the requirements scale. (This could be good for a number of reasons, but it might imply that his advice is only good at that end of the scale.) Still, his teams are using examples where they think it counts. In many applications that are basically CRUD, examples may well be a waste of time.

On the gripping hand, I’m concerned about the potential loss of understanding and agreement between on-site customer / product owner and the team. Examples are the best way to do that and while it is more work for the customer, the examples are more likely to be understood, and to be correct.

Bottom line, I’m concerned about this issue because I like the clarity that results from having concrete tests that are agreed to be “the definition of done”. At the same time, Jim is a smart and experienced person, and we need to pay attention to what he’s finding out there.

This related post just in from Gojko Adzic.

This related post just in from George Dinwiddie.

Jim Explains …

For all our benefit, Jim Shore goes on to describe the things his teams do that help make it safe to operate without Acceptance Tests per se. Good stuff. Read it. I’ll wait.

OK, well, yes. Jim lists essentially the entire contents of the XP lexicon. And all those practices are good and done in concert, you’ll be pretty safe. Let’s look at a few of these for comment.

Jim points out that his teams use unit tests, focused integration tests, and end to end integration tests, written in TDD style. The latter two are exactly the kind of tests you’d get if you could get what I call “Customer Tests”, except that they might not be in a form that the Customer herself could write or read. These should ensure reliability, though they might fall short a bit in communication value compared to Customer-written tests. With good communication around them, this can surely work.

(Mind you, I’m sure it does work. Jim is doing it. I’m just saying that we can see how the laws of software physics support this approach working. It’s readily credible.)

Jim then lists a few million standard XP practices which do support quality. Excellent. Then, he says:

And finally, when my teams find a bug, we fix it right away — or decide that it isn’t ever worth fixing. … After writing a test to reproduce the bug and fixing it, we refactor the code so similar bugs are less likely in the future.

Well. This is interesting. The point is that if we fix things right away, and reflect, and improve our code (and our process I imagine), we’ll improve so as to reduce defect injection. Again, it is easy to see that this will certainly work.

We still have open a small concern about customer-team communication, since all this is mostly on the technical side.

Now Jim goes on to talk about collaboration. He has fully cross-functional, collocated teams, including customers and testers on site. Forgive me for suggesting that if you’re not doing these things, you are not on safe ground for full communication and that the need for Acceptance Tests, however painful, may be increased.

But wait, don’t answer yet! There’s more! Jim’s teams use “Customer Examples” for anything that is difficult to explain. And the developers use these examples for tests … which may not be able to be reviewed by the customers.

This is quite OK. If customers provided Acceptance Tests, the developers would be fools not to run them until they work. It really wouldn’t matter, other than for confidence building, if the customers never saw them run. That has always been true: on C3, our test running person would report the results of the tests every day, and the customer almost never ran any of them. She was satisfied that they were being run and looked at.

Jim goes on to say that he’s OK if the tests are not automated and if they are not customer-understandable. I’m OK that they are not customer-understandable — though I would prefer that they were if it were close to free. I am less comfortable with the notion that they are not automated. My concern would be that if they are not automated, doors are opened to regressions.

It would be interesting to know when these tests are automated, and when they are not, and what other tests are commonly put in place when they are not. Certainly it is not necessary to run every example to be sure that the code works. Probably it is necessary to run some.

Jim then digresses to point out that the real way to trust is to ship stuff that works. Yes. Our question here is about the extent to which Customer Acceptance Tests contribute to things working. I agree that with a sufficiently weird customer they wouldn’t generate trust just by printing “Yay, we work!”

Jim goes on to list every other known XP and related practice as things that his teams do. If your team is doing all those things, then you, too, may not need to automate acceptance tests.

And remember: I hate automating acceptance tests with Fit and suspect that I couldn’t sustain doing them. I also hope and vow that I would do everything else I knew about, to make up for it.

In sum, take a hard look at all the things Jim’s teams do, including every known good practice, especially including root cause analysis on every defect. Notice that his teams do not treat defects as business as usual.

My bottom line on all this is that if I were that good, I could skip acceptance tests as well. On my best days, I’m that good. On my normal days, not so much.

I suggest that the right notion might be this one:

Acceptance Tests are at shu level and the bottom half part of ha level. When your team is at or near the ri level, you’ll have enough other rigor in place to safely drop them. Even then, keep a weather eye peeled for increases in defects. We’re not as good as we think we are.

Summary Remarks

Jim and I, and a few others, exchanged some Q&A on Twitter. My main concern is this one:

Acceptance Tests are what Joshua Kerievsky calls “Story Tests”. They test whether stories work. It appears that Jim’s teams do these sometimes, but not always. They always do lots of low-level TDD, and they do what he calls “integration” and “end-to-end” tests. However, Jim says that they do not always save the end to end tests, sometimes just doing them manually during development.

It seems to me that if any given patch of code is not covered by an automated test, in principle we do not know whether it still works or not. It could be broken by a change to the code itself …  or by a change to any object on which the code relies. Yes, the TDD tests for the objects involved will help prevent defects in the users but they are not as solid a guarantee as we’d have with automated story tests for all stories.

Jim’s observations seem to me to touch on these related but separate topics.

First, customer-owned examples are hard to drag out of the customer. Jim’s teams work to get the examples in the cases where the stories are complex enough to warrant them.

Second, building visible story tests for customers is costly, given today’s tools, and often the team and its customers do not get enough value to make it worth doing customer-visible story tests.

Third, Jim’s teams have found that intensive TDD, including unit tests, integration and some story tests, gives them enough confidence (and enough real quality).

To the first, I’ve seen plenty of teams where the customer (Product Owner) was too busy to create examples. I’ve also seen it lead to slow-downs, and to mistakes, especially when things were complex. Examples are a good way to go in those cases.

To the second, I, too, find the existing tools to be way too much trouble to use. On the C3 project, we built our own tool, which made tests practical. We had examples, mostly from the Customer’s helpers, and it turns out that we reported the results to the Customer: she rarely looked directly at the tests. In addition the C3 project had a very complex domain, which made the tests more valuable.

To the third, I remain a bit uncomfortable. Certainly, if the team is producing very few defects, then what they are doing is working. I would be interested to know some statistics, notably: How often, when a defect arises, could the defect have been prevented by a story test that was not written?

My bet would be that missing story tests will be associated with the majority of story defects that are discovered.

Does that make Customer-owned Acceptance Tests written with Fit or equivalent worth while? No, not if resistance is high enough and value low enough.

Does that make story tests automatically worth while? No, not if errors are infrequent enough.

My conclusion is that certainly what Jim’s teams are doing is working, and they are doing all the XP practices quite well. If other teams do the practices that well, they’ll probably have similar results.

And I think that automated story tests are the simplest and most certain way to prevent defects cropping up in stories later on.

The team returned to the meeting room in better spirits. Kate had drawn a new picture on her tablet.

“Welcome back,” Kate said. “Jim? Susan? Do you have some stories for us?”

Jim took the lead. “We brought five stories, not because we expect five but because we’re not sure whether they are too small or too large. Normally we’d have time to talk with some of the team members to get a sense of size. This time we had to guess. OK”?

Jim and Gil had been leaning forward, ready to leap, but sat back. “OK,” Jim said. “Let’s see what you’ve got.”

“Should we put them all up at once,? Susan said.

Gil said, “Let’s go one at a time and talk a bit about what each one involves.”

The team all nodded.

One story at a time, Susan and Jim described what they needed. After each one, the team discussed how they might test that story, and talked a bit about how the story could be implemented.

When they got finished with the third story, Kate noticed that the team was looking tense. “How is it looking, gang? You look a bit edgy.”

Chuck spoke up. “In the earlier meeting, I was pretty optimistic about getting last week’s five all done. We talked about it during the break, and I’m still optimistic but I think there might be more work than I thought. I feel like this is enough, or maybe even too much.”

Kate said, “Are your stories in priority order, Jim and Susan?” They nodded. “Shall we look at the other two while we’re here, or stop now?”

Jim Anderson said, “I’d like to hear the other two. It won’t take long and it’ll give us a look at the future.” The team seemed to agree.

After all five stories had been discussed, Kate said: “OK. We’re looking at finishing last iteration’s stories and some of these. What do you think?”

Carlos said, “It would help me if we could put the old stories up on the board and task them out.”

Gil said, “We already know what we have to do for those. We don’t need to review.”

Carlos sat back, but Kate said, “I don’t want to be here all day, but let me ask a question, Gil. If we had everyone in the room write down on a sheet of paper what the five other stories are, and what work needs to be done, would all the sheets agree?

“I’m the disappointed story lady, and I really don’t want to be disappointed again, and I don’t want your children and kitties to starve. Should we take some time, or take a chance?”

Gil said, “OK, good point. Obviously Carlos isn’t sure, so let’s review. I’m sure we could figure it out but let’s be a bit more explicit, since we screwed up last time.”

It didn’t take long to put the other stories up on the board and to brainstorm the work that needed to be done.

Kate said, “OK. I noticed that there was a bit of discovery there. We saw a task that wasn’t mentioned this morning, and we found a task that could be done another way. I’m thinking that was worth doing.

“Now let’s take a short break and see what I can look forward to when I come in to the story store next week.”

Commitment time …

“OK, crew,” Kate said. “Looking at these ten stories on the board, what do you think?

No one spoke. After a while, Chuck said, “I still think we are good for the original five stories in two days. I’m sure the one I was working on is only a day, then I can pitch in on something else.”

Jim looked around the room. “Two days, guaranteed?”

Again no one spoke. Then Carlos said, “Let’s play it safe and say three, then shoot for two.”

That seemed to be a good idea.

“OK,” said Kate. “What else can be accomplished?”

The team huddled a bit. Then Jim said, “I know we said three before, but these are a bit larger than we had in mind or something. We’re sure we can do the first two. Maybe the third, but maybe not.”

“Probably not,” said Alice. “All three of these need GUI work that I’ll need to do. I really don’t think it happens.”

Susan wrote on a card, and handed it to Kate with a grin.

Kate grinned. “Good Susan,” she said. “Biscuit later. Gang, Susan’s card says ‘Stretch Goal?’. What do you think?”

“I say go for it,” said Jim. Some of the team nodded, some just sat there.

“Objections?” asked Kate. Again silence.

Finally Jim spoke up. “Kate, Susan and I have been pushing the team to ‘stretch goals’ right along. We think they respond well to a little challenge.”

Kate looked around. “I don’t see smiling faces, Jim. And when you’ve pushed the team to stretch goals, what has happened? Do they meet them?”

Susan raised her hand and got the nod. “I happen to know that. About three-fourths of the time, they fail to deliver one or two stories. When they do deliver, every single time at least one of the stories turned out to be seriously incomplete or broken.”

Jim said, “Yes. Think how bad things would have been if we didn’t keep the pressure on!”

It seemed like the entire dev team snarled. Gil jumped up. “Damn it, you just set us up to fail! We tell you what we can accomplish and then you push us to do more. If we do the job right, we fail. If we do the job wrong, we ship crap. Stretch goals, my ass!”

Everyone kind of fell back. Gil looked around, then said, “Well, sorry. But it’s crap. We can’t win. We can’t even tie.”

Kate said, “Let’s see if I understand. Someone said–I don’t remember who–that under pressure, programmers have secret quality dials that they are tempted to turn in order to make a deadline that should never have been set. The result is code that never comes together, or that seems to work but isn’t really robust. Is that what you mean, Gil? Leaving your seat out of it?”

Gil said, “Yes, that’s it. And it’s crap. It asks us to produce crap and I don’t want to live that way. I’m here to do good work. Damn it.”

Kate laughed. “Well at least you’re consistent about this. What about the rest of you? Do you agree with Gil’s point, if not his format?”

Right around the table, the developers agreed. Chuck said, “Yes. I’m here to do good work. I know what good work is. I really value the clarity we’re getting with these smaller stories. But it doesn’t help me to have work piled on to work.”

Alice said, “Yes. And it’s worse than that. Remember you asked if others could do the GUI work that I do. They nearly could, with a little coaching. But if I have too many stories to do, I don’t have the time to sit with people and help them work through things. That takes a little longer the first few times. Under pressure I feel I don’t have the time and I’m too stressed to do it.”

Kate said, “Jim? Susan? It sounds to me like your ‘stretch goal’ strategy has never really worked. You usually don’t get what you want and what you do get isn’t as good. What do you think?”

Susan went to speak, but Jim held up his hand. “Let me say it. We’re afraid if we don’t keep the pressure on, the team won’t go as fast as they could. They may lay back and take it easy.”

Again the team erupted. This time Kate held up her hand. “Let me take this one.

“How would you feel, Jim, or you, Susan, if Dan told you that he thought you weren’t working as hard as you could, and started pushing more work on you?

“Take a moment, then be truthful. Feel it, and tell us how it feels.”

Jim and Susan paused. Then Jim said, “I’d hate it. Dan knows we’re the best. He gave us these jobs. He should trust us to do the right things and to do them right. Dan’s not like that anyway. Dan would coach us and help us, not treat us like …”

Kate said, “Like what?”

Jim said, “I get it. We’re making the team feel untrusted, and by pushing more work on them, we overload them.”

Kate said, “And why does that overload them? Why don’t they just go along at the slacker pace you’re worried about and deliver what they can? What could possibly explain what they do?”

Susan chipped in. “They’re not good enough …”

Kate stopped her. “No one is ever good enough. Everyone is as good as they are. If they’re getting in trouble trying to reach stretch goals, not being good enough can’t be the real cause.”

Jim said, again, “I get it. They do poorly because they’re trying to do what we ask, and it is too much for them.”

Kate said, “Exactly. And what does that tell us about trust?”

A long silence ensued. Finally, Jim nodded. “They’re failing because they’re trying to do to much. The fact that they’re trying tells us that they’re willing to work hard and that we don’t need to push them. We can trust them to work hard.”

Kate gathered everyone’s eyes. “Is that right, team? Can we trust that you’re doing your best?”

Gil laughed. “Let’s see, we could say no, or we could say yes …

“Yes, you can trust us. We’re professionals. We do this work because we want to do it well.”

Kate said, “One more thing, and we’ll close. Jim, Susan? When the team has fallen short of the stretch goals, how do you feel?”

Susan said, “I feel let down. I feel like they broke a promise.”

The team rustled. Susan went on, “Oh. They didn’t promise. They always said ‘We’ll try.’ Anyway, I do feel disappointed.”

Exactly, said Kate. “I would as well. So when Binary Girl the Feature-buying Lady leaves today, what shall we tell her? Seven, or eight?”

Carl said, “We can do seven. Tell her seven. Right gang?”

Everyone on the team nodded or said yes.

Kate looked around. “OK. We’re all tired. Tough meeting today but I think it has been worth it. The team says they can do seven. Next week BG the SL will be expecting seven. Let’s close for now, but tomorrow I’d like a brief session with interested folks to talk about what we should do to make sure we get seven, and what to do if it turns out we can’t. Would 9 AM work for that? Attendance is optional.”

Everyone agreed.

“OK, seven stories it is. Thanks. See you tomorrow.”

The team filed out. Kate gestured Susan to stay. “Susan? I’d still like to have a session with you. Today? Or is tomorrow better after all this?”

“Do we have to?” said Susan. “I’ll be good.”

“Yes, please,” Kate said. “I just want to help you understand what I’m up to.”

“OK, tomorrow then. After the 9 AM thing?”

“Great,” said Kate. “Don’t worry. I promise not to blow my whistle.”

“Yikes, I hadn’t even thought to worry about that. OK. I’ll just stay after the 9 o’clock, OK?”

Kate said, “That’ll work fine. Thanks. Bye for now.”

Shop Class as Soulcraft: An Inquiry into the Value of Work
– Matthew B. Crawford

Crawford speaks eloquently about the value people feel in work that is tangible. He is primarily extolling the joy we feel in applying the manual arts well, and suggests that we should value those skills more highly.

He points out that a craft like plumbing is not subject to off-shoring, because it must be done on site. The Agile software community, of course, understands that working on site is far more effective for us, though it is not logically necessary in quite the way that plumbing is.

But there’s more for us in this book besides a good read. People do get joy and work harder, when their work is tangible. This has implications for how a developer can work toward greater joy, by focusing on delivering real features that people want.

There are implications for management as well. Because people are motivated to see their work come to fruition, we’re likely to get better results with feature-focused teams rather than teams divided along other lines, such as infrastructure levels.

This book is a quick and enjoyable read and it will give you valuable ideas.Give it a look!

If your organization is like many others, you have adopted Scrum or another “Agile” method. You have received some benefit and are glad you have gone Agile. Yet very likely you feel there should be more — and you’re right!

The creators of Scrum agree: Ken Schwaber has said that perhaps only twenty-five percent of Scrum teams get the full benefit of doing Scrum. Jeff Sutherland has said that every high-performance Scrum team he has seen was using practices that go beyond Scrum.

To move beyond what Scrum and “Agile” have given us, we need to do some new things, and likely we need to do some old things better. Let’s explore together how to recognize those things, and how to begin to address them.

We’ll be looking in these articles at the things that need to be done inside your projects, and outside your projects. We’ll point out what the manager or executive should ask teams to do, and we’ll talk about why. We’ll also be look at what managers and executives themselves need to do to help projects to become successful. Some of these ideas may not be new to you, and others probably will be. Some of the ideas will be challenging, even a bit difficult. Make no mistake: to attain the higher levels of performance, we all need to work a bit differently.

The fundamental benefit that the Agile methods try to provide is the continuous delivery of “Running Tested Features”. Here’s why:

For most projects, we have a deadline in mind, and we have some understanding of the capability we want to have by that date. There are two key questions we need to address:

Planning: How can we decide how much time and how many people we need, in order to get the capability we need by the time we need it?

Managing: Given this overall plan, how can we manage the project to get what we need by the time we need it?

Even with a great plan, we can’t succeed without good project management. We can even produce great results from poor plans. Our teams need to execute every day, and we plan only seldom. So first we’ll look at ongoing management of our projects, and then at how we should plan them.

Conventional software project plans used to look like the picture above. Mysterious and magical things go on in development, and then toward the end of the project, we expect to see features rapidly coming on line, with a successful delivery right at the deadline.

Yes, and on Christmas, Santa will bring you a pony. When we plan this way, the reality usually looks more like this:

The mysterious and magical stuff happens all right, but when things start coming on line, they start late, and arrive more slowly than expected. The project turns from “ALL OK” to “IN TROUBLE” very late on. As managers, we have little or no time to respond to the trouble, yet our sunk costs are so large that we have no real choice but to go ahead.

Incremental development, as Scrum and Agile taught us, gives us the ability to manage the project. The capabilities of the modern project come on line incrementally, a few every month. We can project the growth of features to see how we’re doing. Even if we don’t go any faster, we’re in a better position to make decisions about what to do. We can look at feature growth and the deadline and decide to what extent we should cut scope, and to what extent we should extend the deadline. The incremental project looks like this in comparison with the conventional:

Because of the value of this information, the modern software manager asks for an incremental process, with smooth growth of features. We demand that actual progress be faithfully presented to us regularly, like the blue line above. Our first job is to recognize that a faithful blue line is the truth, and that our job is to manage to that truth. We build our organization’s plans around the truth of what is happening.

Secondarily, we do want to help that line to climb more rapidly. We do things to help it climb – and we do things to make sure that the line remains true. As we succeed in improving feature growth, we update our plans. Always, we focus our plans on what’s true, not on what we wish were true.

In the articles that follow, we’ll discuss what it takes to keep that line growing smoothly and steadily. We’ll talk about how to manage them, and about what the people on the projects need to do. Our focus will always be on this blue line of progress, keeping it smooth and steady, so that we can guide our projects to the best possible success.

Teams using incremental and iterative processes have daily stand up meetings that provide an opportunity to observe how things are going, and they can set up a quick meeting to remove obstacles and improve things. At the end of every iteration, there is an opportunity to review the whole iteration and decide how to make things better.

Most experts, and most named iterative methods, explicitly call for a retrospective, at the end of each iteration if not more frequently.

Some people — and at least one named method — argue that if we iterate, reflect, inspect, and adapt, we can and will become a high-performance team. This is true, to an extent: if we observe the right things and make the right choices, then of course we can improve as much as we want. Unfortunately, we were not born knowing the right things to observe, much less the right things to do about what we see. If we get it wrong, good things will be missed.

Alpha Team has been building features for quite some time now. When they hand things off to QA, they rarely come back, and when they do come back, Alpha looks at what they’ve done, and devises ways to avoid that kind of return from QA. Alpha has beefed up its testing ability substantially, and they use pair programming and code reviews to great benefit. Still, things seem sluggish. Even with no bugs coming back from QA at all, it still takes a week or two to be sure that features are done. This seems wrong, but they don’t see what to do.

Alpha Team, for some reason, has not thought of some valuable ways of restructuring the team, and their work, to help them get things done more quickly. The most effective teams move the QA function before development to the greatest extent possible. The testers are part of the team, not in some separate group. They provide tests before the iteration even starts, and the team considers anything “done” that passes all those tests. When something passes all the tests but does not seem “done”, the whole team considers, not who screwed up, but what tests were not thought of, and how to improve their thinking for next time. The result is much more certainty of where the team stands, and much less stress all around.

Where is a team supposed to get ideas like this? If we wait for each team to think of things, some of them may never do so. If we set up a standard way of doing things, we’ll be cheating ourselves of learning, and we’ll wind up making process followers of our people rather than intelligent creative workers.

What’s most important is that team members need to improve their skills. The organization can help with this by providing training, by subsidizing books, or by other means. We believe that the best approach is for the organization to encourage retrospectives, draw proposals from the team, try them, and watch what happens. When ideas come in from outside, make a point of noting what book or web site was read or what course was taken. In short, show respect for people’s efforts to improve themselves.

Sources of Good Ideas

For internal technical practices, we suggest looking at the practices of Extreme Programming. In particular, automated testing, both Test-Driven Development and Acceptance Test-Driven Development are very valuable, as we discuss elsewhere.

For in-team workflow practices, look to Kanban and other approaches to limiting “Work in Progress”. These ideas highlight bottlenecks and delays inside the development team.

It’s important to look at the larger flow as well. Nothing good will happen if we get development time down to zero in an organization that takes six months to get an idea off the ground and another six months to deploy it. Here we would recommend ideas from Lean, such as Value Stream Analysis.

Is Our Process Framework Complete? Is It Sufficient?

If your process framework includes taking in features at the beginning of an iteration, delivering them complete at the end, and reflecting on and improving what you do, it is probably complete. In principle, that’s enough to make a good framework.

It’s far from sufficient as a process, however. There are many practices, many ideas, many things that we have to do in order to improve. Those things are necessary and they should come from the team and its explorations. No process framework can list everything we need to do. No framework should try.

There has been some discussion of late regarding the use of Test-Driven Development ideas in non-software areas. This is a really good idea and I would like to talk about it here.

Kent Beck introduced “Test-Driven Development” as part of XP. This is a developer-focused design technique whereby developers write  microscopic tests and make them run, in a very short time cycle. The TLA is of course TDD.

XP proponents also introduced the notion of expressing requirements in terms of executable tests that the Customer (PO) understands, used to determine (in part) whether the software does what was requested. These are often called Acceptance Tests or Customer Tests, and facilitate communication of backlog items in concrete terms. By analogy, this technique is called ATDD: Acceptance Test Driven Development.

It’s worth noting that these two approaches, TDD and ATDD, are essentially the same idea: provide a concrete criterion, fulfill that criterion, then repeat. The distinction is of value because the idea is used at two very different levels, a matter of minutes for TDD, and days or longer for ATDD.

Concreteness and clarity are always valuable, even when what is being done is not amenable to executable testing as is software. Can we perhaps extend this test-driven idea to other areas? Certainly we can and probably we should.

Wise product owners will always make their needs as clear as possible. One way of helping teams focus on this is to extend the Customer Testing notion to non-software situations. In these other situations a Product Owner test might be as concrete as a set of measurements that a piece of hardware or furniture must meet, or vague as a set of criteria for hiring a new team member. In all cases, clear communication is enhanced as the criteria are made more clear.

Similarly, we might apply the idea of providing concrete criteria between elements of the dev team, or even for a single dev team member. For example, if a dev team member was building a stairway,  he might find it valuable to build a jig or template to ensure that each step was equally spaced and sized. Carpenters have known this trick for centuries, but we can certainly see that it is an analog of this more modern “test-driven” idea.

Where is the chicken, and where is the egg? We probably don’t care. The basic idea of setting up concrete criteria for the construction of something we want, and checking those criteria as we go, has been around a long time. Two specific instances of this idea have been given the names Test-Driven Development (TDD) and Acceptance Test-Driven Development (ATDD) in the software community, and it’s probably worth reserving those two terms for software, to avoid confusion.

The basic ideas, though, have been around for a long time, and are of value almost everywhere. Let’s use them where they can help us.

Susan Prior drops in on Kate to talk about some new products she would like developed.

“Hi, Susan,” said Kate. “You wanted to talk with me?”

“Hi, Kate. Yes. I’ve got some product ideas that I think we should invest in, and I wanted to talk with you about getting started. Dan is interested and said I should get your input.”

“Sure. What’s up?”

“There are three new products I’ve been talking about with my people and with customers. They’re all getting good reactions from prospects we’ve talked to. I think they would each bring us about a million dollars in the first year after release. I can show you the figures.”

Susan and Kate went over the figures, and talked about the three projects, which were codenamed Emerald, Obsidian, and Ruby.

Kate said: “The revenue looks good for each of these. What do you know about costs?”

“Gil and Jim looked at the specs and agreed that each one of these is about 15 person-months to develop.”

“OK,” said Kate. “What are you thinking you’d like to do?”

Susan said, “I have three people in Marketing who are working up feature stories. I would be the official Product Owner on all three, at least through first release. Development can free up six people soon. I’d like to get those six people committed to these projects for nine months to a year to get the products out.”

“Hmm. How did we get from 45 person months to 72 all of a sudden,” Kate said. “That’s a lot of money, from 375,000 to 600,000 dollars.”

“Well, I figured in a little extra in case Gil and Jim under-estimated, or in case they are late.”

“I understand, though I’m not sure I agree. And why are you going for all the money and time for all three projects at once?”

Susan said, “These are all important and valuable projects. We should get them all under way right away.”

“You’re asking Development to work on all three of them at once? Two people on each or something?”

“No,” said Susan. “Gil and Jim say the critical mass for each product is five or six. So I’ll have the team work on all three products.”

Kate put on her confused face. “All three? Wouldn’t it be better to work on them one at a time? Most important first?”

Susan said, “No. They’re all equally important and equally valuable. They should get equal priority.”

“Equal? Are you sure?”

“Well, as closely as we can tell,” said Susan. “Each of these projects takes about the same amount of time and will bring in about the same revenue flow. So they should get equal priority.”

Kate said: “Let’s test that thought. What if we could only afford one? What should we do?”

“I knew you would do this. This is my chance, and you want to wreck it. You’ve always hated me ever since the beginning.”

“Calm down,” Kate said. “I don’t want to wreck anything. And I don’t hate anyone. I can’t afford it, it poisons my soul. I just asked a question. What if we could only afford one? What would be best to do? Would you like to take a break to think about it?”

Susan steadied herself. “No, I’m OK. I’m just so excited and scared about this. These ideas are really good. They could make a difference to the company and I can show what I can really do.”

“I’d like that, too. You got off to a bad start with the team on Empire. But I know they are willing to work with you on this,” Kate said.

Susan asked, “You know? How do you know?”

Kate laughed: “I know almost everything around here. People tell me things. I think it’s because I spend so much time in the coffee room.

“Gil and Jim have both said, several times, that they respect your ideas and would like to see them get implemented.”

Susan calmed down some more. “They did? They never told me.”

Kate laughed again. “They’re programmers. They never say anything personal to anyone. It’s their job.”

“Good point. Anyway, if we could only do one, it should be Obsidian.”

“Why Obsidian?”

Susan said, “Three reasons. It’s a bit simpler, the prospects we have talked to seem more enthusiastic about it, and we understand it better.”

Kate thought, then spoke. “Those seem like good reasons. Thanks. Now, thinking again about doing these as you propose, let’s look at some pictures.”

This time Susan laughed. “I knew I could never get out of here without pictures.”

“It could be worse. I could make you listen to me singing. OK. Let’s look at a nine-month plan.”

“OK,” Kate said. “Here’s a planning area. We have the three projects, Emerald, Obsidian, and Ruby. They’re all about the same size in stories, and you’re planning an ideal nine-month project. I won’t show any contingency here, just for clarity. OK?”

“Yes,” said Susan. “We may need the contingency, though.”

“Don’t worry, we’ll talk about that later. Now here is the way features will grow over the nine months, according to plan.”

Kate added feature lines to the chart:

Kate went on. “The features all grow at about the same rate until the projects are all done after nine months. That’s the starting picture, agreed?”

Susan nodded. “Yes, exactly. The developers would work in parallel on all three products”

Kate said, “In parallel. Why is that good?”

“So we could see progress on each product.”

“And why do we need to see progress on each product?”

Susan thought. “Well, so that if anything goes wrong we can do something about it.”

“OK, I guess. Now when do we start making money from these products?”

Susan said, “If it goes according to the plan there and we don’t need the contingency, we start making money shortly after the nine months is up.”

“Shortly after,” Kate said.

“Well, we would have to roll them out with advertising, documentation and training and such.”

“Those couldn’t start sooner?”

Susan said, “Well but what happens if  the developers are late?”

“You keep asking that,” Kate said. “In my view developers are never late. Products are late sometimes, but never developers. I suppose that sounds like crazy talk.”

“Frankly, yes,” said Susan.

“OK, we’ll defer that for a moment. Returning to the picture, we start earning money at three million a year, starting nine months out, is that right?”

“Yes, that’s right,” Susan said.

“OK. Now take a look at this drawing instead.” Kate erased some lines and drew some new ones:

“Susan, now I’m drawing a picture of what happens if we do the three projects one at a time. The new vertical lines are three release dates, at three months, six, and nine.”

Susan said, “But what about contingency?”

“I’ve left that out, as I did in the previous picture. We’ll talk about it though, don’t worry. Now in this scheme, we focus the developers on first Obsidian, then Emerald, then Ruby. Look what happens to Obsidian development:”

Susan said, “Yes. It gets done right away. But the others aren’t even started. That’s risky, isn’t it?”

Kate said, “I’m not clear on why it would be, so marshall your thoughts. But tell me what happens to company revenue with this scheme.”

“Well, we start earning after three months instead of nine.”

“Do we make more, or less, or about the same,” Kate said.

“About the same. Maybe more, because we are in the market sooner and might get more early adopters.”

“What about the impact on this fiscal year, and on our ability to fund projects?”

Susan scratched on a scrap of paper for a moment. “Well, we’d be to market six months sooner and in the first six months I think we get about 40 percent of the 12 months revenue for Obsidian. If we start right away, that would all fall in this fiscal year.”

“Yes! That would be $400,000 and that’s just about the low-end cost estimate for all three projects. What does that do to our concern about deferred risks for Emerald and Ruby?”

Susan thought. “OK, I see. Total cost of those two projects is scheduled to be less than $300K. Even if they completely failed, Obsidian’s first six months revenue would cover the costs. But again, what if development can’t get Obsidian done in three months?”

Kate smiled. “Go with me a bit longer while I finish this picture. Here’s the picture of the other two projects getting done sequentially after Obsidian:

“So here I’ve just assumed we’ll do Emerald next, then Ruby. It could be done the other way. So Emerald is done after six months. What happens with revenue there?”

“If things go according to plan, we think Emerald would bring in about 150K in its  first quarter out,” Susan said.

“Yes. So even that revenue is a big chunk of the cost of doing Ruby, and of course we’re already covered by Obsidian. So by the end of the three projects, we would see $500,000 in revenue, and it would all fall into this fiscal year. And the total cost of the projects you are asking for is around $375,000 if we use the development estimates. Even with six developers, we are looking at expenses of about 3/4 of 600K, or 450K for the nine months. See the difference?”

Susan stared for a moment. “Yes, sure. Done your way, the same projects actually break even this year or earn a small profit. Done my way, we spend $450,000 before we get anything.”

Kate said, “Let’s not think my way your way. Done in parallel, the projects burn through almost a half a million dollars this year. Done in series, they burn $150,000 in the first quarter, then start earning back, and we are back to even by the end of the fiscal year.”

Kate smiled. “If it was your money, Susan, which way would be ‘your way’?”

Susan said, “I get it. My way is to do this in series. That’s what I recommend. Yes, totally. I’m sure that’s what I meant all along. Serial.”

Kate laughed. “I thought so. It’s almost lunch time, are we done for the morning?”

Susan thought. “I think so, yes. I’d ask to have lunch with you but I need to think a bit first. I’m still worried about contingencies and risks.”

“Good,” said Kate. “I’d like you to think about that. As a hint, think in terms of what you can do as Product Owner to manage the risk. But keep in mind it’s your money, and you can just barely afford the $150K a month for your six developer team.”

“That’s scary,” Susan said. “I guess that’s what you feel all the time?”

“Well, it’s the problem I have all the time, Susan. But I’m not very scared. See if you can figure out why. When shall we meet again?”

“I’d like to come in again tomorrow, if that’s OK,” said Susan.

“Sure thing,” said Kate. “I’ll be here. Thanks, this was fun.”

“It was,” said Susan. “Oddly enough, it was. Thanks, Kate.”

“You’re welcome, Susan, and thank you. We could use another three million in revenue.”