The Nokia Test (6): Estimates created by the Team

Another installment on the Nokia Test.

Before we begin, a quick mention that Jeff Sutherland has done an improved scoring on the Nokia Test. See here.

So, the next item on the test says: "The Product backlog has estimates created by the Team."

Why is this important and what does it mean? Meaning first.

So, normally in Scrum estimates mean estimates of relative size/complexity in Story Points. See Mike Cohn's book:


Each story (or at least any story in a Release Plan or Product Roadmap) needs a story point estimate. You can't bring into Sprint Planning any stories without Story Points. No, no, no. (If the Story is discovered in Sprint Planning, that's a bit different.)

And these estimates are created by the Team of implementors. Those who will do the real work.

Why?

Well, estimating my own work gives me much more motivation. And responsibility. And a reason (well, another reason) to appreciate how my work interacts with the work of others. If the Story Points come from some other person or group, they don't have the same feeling.

Yes, there is a downside. Not every Team is equally good at estimating. Yes, it's true. But we are convinced that the negatives are more than offset by the positives.

Another key reason for this test is how we will, later, use Story Points to know velocity. And how we will use velocity for many things, from which I will highlight three:

• Tell some managers: "Look at our velocity. We are going at 20 work units per Sprint. Stop hoping and dreaming that we will go at 40 work units. It's magical thinking and it ain't happening. And your trying to make it happen is just making things worse."
• Tell ourselves: "Folks, we're going at 20 Story Points, but we need to do better. Let's identify a top impediment and FIX IT. So that we can go 22 or 25. Now! (But not by working harder.) "
• Face the truth. Velocity is a way to help us all face the truth. And take action. (Ex: If you name an impediment, someone is going to ask: "Ok, what do we do about it?")

OK. That's some commentary to start with. Now put your thoughts into action.

The Nokia Test (5): A prioritized Product Backlog is essential

We started a series on The Nokia test some time ago. This is the fifth explanatory post about the test. To find the others, search above (left). And here is the original post.

The second item in the second section of the Nokia Test is this:

• There is a product backlog prioritized by business value

Seem obvious? Well, maybe you don't want to say so quickly.

First, if we are doing Scrum, we must have a Product Backlog. A Product Backlog is a list of all the work items the team is expected to work on. If the Team is doing software, most of them will be features at a high or low level of granularity. In Agile, the growing preference is to put these Product Backlog items in User Story format.

This part of the Nokia Test does not require you to use the User Story format. Or any particular format.

"Prioritized": Ummm. What does that mean?

Well, I think it means that the Product Backlog has been put in order of Business Value.

The Nokia Test does not give us any hints about what Business Value is. My opinion is that this itself is a complex topic. But I think the implication is that the Product Owner (who owns the Product Backlog, and makes all final decisions about it)...the Product Owner will put the items in business value order.

Does the test imply that Nokia always wants to work the highest business value items first?

This is open to some debate. My view is that one can still pass the test if one also uses "cost" (or story points as a proxy for cost) and dependencies/risks as additional factors (bits of information) in helping the Product Owner to order the work.

Let's put this some other ways.

The technical team does not have the final decision about what order to do the work in. Sometimes it is better to be inefficient and get some high Business Value item(s) out there in production.

Since all Product Backlog items are not of the same size (let's call this "amount of work" for now...although one must tread carefully here), and not of the same Business Value, the Product Owner must do cost-benefit analysis to determine (implicitly at least) business value per unit of work for each PB item.

Why do we prioritize by business value? We are always trying to discover and release business value (eg, increase customer satisfaction). Quickly. Only if we use Pareto's 80-20 rule can we do the most important stuff first. Ordering the PB items puts Pareto's rule into play.

And I think the Nokia Test says the Product Owner cannot cop out and say each PB item has the same amount of Business Value.

Similarly, this part of the test does not prohibit some team members (developers), where they have useful knowledge, from influencing the Product Owner about business value and about the order of the work. (Still, Scrum says the Product Owner gets the final say.)

* * *

Well, that's a start on the implications from this one part of the Nokia Test.

Please see our other posts on the Nokia Test.

Note: The picture of the story card above was stolen from here: http://www.pragmaticsw.com/newsletters/Newsletter_2008_05_SP.htm

The Nokia Test (4): You know who the product owner is

In this series, we are going over each question in the Nokia Test.

The first section of the Nokia Test is a quick determination: are you doing incremental development? Then second section is: are you doing Scrum?

We are now up to the first question in the second section is: Do you know who your Product Owner is?

Clearly, it is not just "do you know his or her name?"

The Product Owner has these responsibilities in Scrum:

• Defines the features of the product, decides each release date and content – gets product backlog ready
• Is responsible for the profitability of the product (ROI)
• Prioritizes features according to market value
• Can change features and priority at next Sprint
• Accepts or rejects work results

A few comments.

The Product Owner is the main voice of the business side into the Team. She is responsible to assure that the team understands everything the business can tell them about the effort (high level and low level).

The Product Owner is the main risk manager, since most risks are business risks, and must be managed as trade-offs against other things (values, risks, etc). At the same time, the team members are seen as business people also (they signed up to deliver business value), so in some ways managing risk is a collaboration. But when the final decision must be made, she must decide (at least, if it is decided within the team).

The Product Owner is part of the team. (This was debated for awhile in the Agile/Scrum community; the best advice now is to treat the PO as part of the team.)

The Product Owner also has outward facing responsibilities. Most of the team members work within the team, in most senses of that word. She is also responsible for understanding the customers (eg, end-users). This takes constant work, since the customers are always changing.

The Product Owner manages the stakeholders. The stakeholders are people outside the team who have a stake or a say in the product being created. Maybe the product is mainly for external customers, but operations must use it also. Maybe Legal or Compliance has some say, etc, etc. In large corporations, these stakeholders take a lot of work (or so it seems to be required). A key area in managing the stakeholders is getting down to one prioritized Product Backlog, at least the Product Backlog items for the next Sprint.

So, how much time does the Product Owner spend with the Team? There is no precise answer to this question; it depends and it varies. But the Product Owner should work with the Team at least as much as the Team wants. And the Team should want the PO as much as having her will improve the product (speed, accuracy, more value, better, cheaper, etc). It is seldom that one can learn too fast or too much.

The typical situation I find is this. The Product Owner person and the Team start out not used to working together. And not seeing a lot of value in working together much. But they learn about the value over time. Toward the end of the first effort, the PO will usually say "Wow, this takes a lot of my time to be with the team, but it pays such great benefits! It is well worth the effort."

Perhaps this Nokia Test item should read: "The Product Owner and the Team collaborate well"

Two cheers for the Nokia Test (2)

A comment by Kelly Waters and a discussion yesterday with Peter Smith prompts me to add another comment.

Some are attempted to come up with a long list of all the practices involved with Scrum (or Scrum plus other parts of Agile). And then score each team.

I don't think the long list replaces the simple short list. First, for many people, the simplicity is important and necessary. They need a quick, bright line to guide them back on to the road as they start to veer off. The short list does that better.

Regarding the long list...

I have done this (or was required to do it), and found one aspect of it very useless and another aspect very useful. And I just thought of another useful way to use the tool.

Useless: I think generally scoring each team is not very useful. Do I care that one team is 63 and another team is 72? You might do it, but I would not make too much of the number. Each practice is not equally important (and probably the importance varies by situation). I think it would be useful to tell the team "well, you answered 'yes' on 30 out of 40 of these items...what does that tell you?"

By the way, we found it useful for an external coach to walk through the long list with the team (ie, not the team's ScrumMaster/coach).

Useful: Asking the questions, some as yes/no and some with a sliding scale answer, is useful in generating great conversations with the team. The trick is to help them identify one weakness or root cause as the highest priority impediment, and to act on it. ("We need a half-day Scrum refresher course, with a focus on the principles" might be one example.)

Useful 2: I think it might be useful to pull data across many teams, and determine what are the practices that teams generally are doing least. That tells me the "Scrum Center" (if you have one) has done a poor job in communicating the value of the least used practices. Or at least that is a likely root cause. Then the Scrum Center can consider how to communicate why that practice is indeed valuable.

I hope these are actionable comments for you.

Two cheers for the Nokia Test

I am an advocate of the Nokia Test, up to a point.

Why do I like it? I think it is a simple way to set some sort of lower boundary on Agile (Scrum) and it tends to make two problems more visible: Cowboy Agile on one side and Agilefall (aka Wagile) on the other side. Cowboy Agile is where you are doing stuff you are making up on the fly (mainly not doing things you personally don't want to do). Agilefall is where you are doing Waterfall (or mostly waterfall) and calling it Agile (or Scrum). [Alert: Those are my definitions of those terms; others may proclaim somewhat different definitions.]

If the Nokia Test causes people to say, "oh, gee, maybe we aren't really doing Scrum", I think that is a good thing. Especially if they think about it, and decide to do Scrum better.

As with almost anything, it can be misused.

As one example: If people use it as just a checklist, and say "oh, we have 'em all checked off; so we must be doing Scrum well", that would not be helpful. In my opinion.

The Nokia Test only defines an absolute minimum set of things regarding Scrum.

Use the Nokia Test to start some conversations around "Are we really doing Scrum and really getting value from it, or are we just playing at Scrum?"

Passing the Nokia test does not say you are doing Scrum well. Among other things, to do Scrum well, you must really understand the principles and the values of Agile. So, for example, the Nokia Test does not talk about principles. The Nokia Test, in my view, does not even cover all the important practices.

So, it is a test more to determine who is NOT doing Scrum than who really is doing it. It establishes a lower guard rail; if you touch it, you should think a bit.

What does it mean to fail the Nokia Test? Well, whatever you are doing, you should stop calling it Scrum. Does failing the test say you are "bad"? No. Does failing mean you are less productive than before? Not necessarily in my opinion. BUT...although you might be more productive than your waterfall days, I think you are likely to be quite clearly less productive than you could be if you followed Scrum a good deal more closely.

One final thought. I really think using Scrum well can make your teams a whole lot better. But the real point of Scrum is not to brag "I'm doing Scrum just as they told me", but to say something like "Scrum is helping us produce a lot more business value each month....". You should want to do purer Scrum to get more business value, not just to be a purist for its own sake.

The Nokia Test (3): Agile Specifications

The third line in the Nokia Test is: "The Iteration must start before the specification is complete."

What does this mean?

The first practical goal was to eliminate the analysis paralysis and delay associated with waiting until the specification was "complete". I don't know all the details at Nokia, but I have lived them at many other firms.

What is wrong with this old waterfall process? (at least in my opinion):

• too much delay
• a pretense that further change (or learning) won't happen
• a magical belief that the customer can really fully understand a spec (I have seen it happen maybe once)
• a magical belief that "all the questions are answered by the spec", when we know that people learn much better in a face-to-face Q&A format
  

As an aside, anyone who has made Waterfall succeed of course does not rely on the spec alone; we use agile-like conversations and other agile-like tricks to make the meaning clearer.

So, what are we advocating in Aglie (Scrum) to replace this broken process?

Well, it turns out to be a Goldilocks idea. We have some wish to make the team efficient and at the same time we know we are learning all the time. So, we advocate an Agile Spec. Not too much, not too little; just right.

What does this mean?

Well, at a minimum it means that the business person involved (in simple terms, the Product Owner) at least thinks he understands the story (let me assume the team is using User Stories). And at least thinks he can answer all the questions. It is also a good practice for the Business Analyst (or someone) to write down a page or two or more of notes. In the course of doing that, she (the Business Analyst in this example) will ask the PO several questions, and find out that he doesn't have the answers yet, and so new learning will happen.

But the Agile spec is very short. Maybe a bunch of diagrams. The good stuff is not buried in wads of paper.

Who decides what the Agile spec looks like for a given team? The consumers. Primarily the coders, the testers and other team members who must use it. Different projects and different teams (and even different situations) may require somewhat different Agile specs.

Let me be clear. The Nokia Test does not say "use an Agile Spec". I (and others) are recommending an Agile Spec as a best practice.

And Scrum does not say "use an Agile Spec". Scrum does say "improve your engineering practices" and I (and many others) would suggest that one improvement area would be around "requirements", and more specifically, one would be using Agile Specs.

Be aware that some in Agile would advocate no written spec at the beginning of the Sprint. Nothing written; just have conversation in the Sprint. This has worked, although I think it typically is not optimal (ie, the team usually could have done more if they had used an Agile Spec). While I agree with the importance of the conversation, face-to-face with Q&A, I also think the 1-5+ page Agile Spec per Story is also helpful. As long as there is discussion between the writer and the readers about what in it was useful or in the way, or just missing (and needed to be written down). (Answer to a question: Yes, all the Agile Specs developed so far could be in one document, if the team found that useful.)

I would suggest that about 5% of the team's total time in this iteration should be used to prepare for the next Sprint. This includes building and discussing (at a high level) the Agile specs for the next Sprint.

Remember that we are always learning. Just because something got put in an Agile spec does not mean it can't change (if we now know better). At the same time, an unprofessional attitude about learning ("oh, I'll just let myself learn about that later") is not allowed either. We are trying to learn as fast as we can. By putting all our minds together.

{Note: To find our previous posts on The Nokia Test, you might search on that term in the Blogger search box at the top of this page. We have 3 earlier posts.}

The Nokia Test (2): Working Software

The second line in the Nokia Test says: Software must be tested and working by the end of each iteration.

This is the second of three items that confirm that the team (project) is "iterative". Then there is a series of small tests (within the Nokia Test) for whether the team is really doing Scrum (in Nokia's opinion).

Why this second line (element)? As with almost everything in Agile, there are many good answers to that question. I will highlight three.

1. We can get better feedback. Only by having the software tested and working, can the Product Owner and the Stakeholders give the best feedback. And we want short, small, fast feedback: "Yes, it's what I said, but now that I see it, it's not what I want." When it works, and they put it together with everything else that is working so far, then they can lift their eyes up from the weeds, and start to see if a real customer product is starting to emerge (be formed). Sometimes this allows them to creatively discover other visions for the product (or improvements to the vision of the product).

Arguably, one could get feedback without being fully tested. I am not particularly impressed by that argument, but I'll leave it for now. But my next reason for this line in the Nokia Test addresses that argument.

2. Working (fully tested) software is the primary measure of progress. This is straight from the Agile Principles that were agreed when the Agile Manifesto was written.

And why is that important or right? Well, before that, many were measuring progress by how much paper was churned out, or how many detailed tasks were done, or by the dev team saying "We're 63.2% done". None of these were ever very reliable (at least in my experience and that of many others). Certainly they had minimal meaning to a business side person who had to manage the risk of delivery by a specific date.

OK, so what does it really mean to have working, fully-tested software? Well, each team must define at some level of detail what "done" means. A company, at a slightly higher level, might also have a standard definition of done (with perhaps some wiggle room for special cases).

That definition of done would typically include (or at least address) things like:
* coded (duh!)
* automated unit tests built, in the configuration management system, run, fully passed
* refactored (anything that needs to be refactored has been: code, design, arch) Note: some refactoring might also occur after some things below.
* put into the integrated build and a new (QA?) environment (the new story does not break other things, etc.)
* automated functional tests built, in the CM syetm, reviewed by business guys, fully passed per a QA person
* other testing done (more variable by effort...eg, some performance or exploratory testing)
* business side testing and review (maybe by the Product Owner...full thumbs up)
* fully documented (any docs that need to change because of this story have been changed and reviewed and are perfect)
* no outstanding bugs (or none of any consequence)

If a story passes the above criteria, then a business person (in most projects) can assume a fairly clear and small amount of additional effort to take that story or feature live. This knowledge can be very powerful and give the Product Owner the courage to identify more early releases.

3. Working and fully tested software is necessary to know (meaningfully) the team's velocity. (Velocity is really a later element in the Nokia Test, but this line in the test is setting up the team to have a meaningful velocity.) Velocity is useful in many ways, but I'll just explain it with the family vacation metaphor. When the kids in the back seat ask "when will we be there?", if I know we are going 60 mph (our velocity), and it's 180 miles to go, even I can give a pretty accurate answer. Good enough to make mission critical decisions like whether to pull over for a potty break. And, as it turns out, good enough for most real business decisions. And, as it turns out, giving us about as info with as much quality as we can get.

The Nokia Test (1): Iterations must be timeboxed

I will be doing a series of posts that discuss each element in the Nokia Test (see earlier post). In this first post, we will focus on the first element in the Nokia Test: "Iterations must be timeboxed to less than six weeks."

First, remember that the first section of the test is to determine whether a team is iterative. (The second section determines whether they are doing Scrum.)

This first element, the length of the iteration or sprint, in standard Scrum according to Ken Schwaber is one month. There are many Scrum teams now doing 2 week sprints. Or even less. Note: One version of the Nokia Test that I have seen says iterations 6 weeks or less. This is a standard in iterative or Agile, but not in Scrum (which is 30 days (4 weeks) or less).

The iterations are time-boxed. This means that the length of the iteration does not change from iteration to iteration. And we do not extend any single iteration (or sprint) because "we're not quite done yet".

Why are time-boxes important? First, "when a man knows he is to be hanged in a fortnight, it concentrates his mind wonderfully." (Samuel Johnson) It is easy for us to get distracted, and the time-box forces the team to face the real world. It forces them to cut through analysis paralysis.

Time-boxes are also wonderful is a slightly different way. You are no doubt familiar with the Pareto principle (aka the 80-20 rule or the law of the vital few). So, the team is forced to choose those "20" most important things to do and get done in that time-box, out of the wonderfully long list of "100" good things to do in their lifetimes.

And, by making the goalposts immovable, the team starts to see that the time-box has meaning. They must estimate better or work better or in some other way improve if they want to complete their work consistently every iteration.

The time-box also enables the team to reflect, on both their work product and on their work methods and approaches. And to get feedback, and make mid-course corrections. This feedback mechanism is not stated specifically in the Nokia Test, but to me the feedback is in there because it is such an important part of Agile and of Scrum.

We will come back to the usefulness of the time-boxed iterations as we discuss other parts of the Nokia Test. While, for the sake of small blog posts, we are looking at each element, it is really when the elements are together that the test starts to have real power or meaning.

The whole is greater than the sum of the parts.

The Nokia Test

Nokia (the cell phone maker) uses Scrum. They have developed a test to check whether a team is really using Scrum or just doing what I call Cowboy Agile (see wikipedia on cowboy coding). Or doing Agilefall (talking Agile terms, but really doing mostly waterfall).

The Nokia Test is in two parts.

First, are you doing Iterative Development?

• Iterations must be timeboxed to less than 4 weeks
  
• Software features must be tested and working at the end of each iteration
• The Iteration must start before specification is complete
  

The experience is that if you ask a lot of "Scrum" teams if they can pass this part of the test, they can't. If you are at a conference, often not a single team in the room.

The next part of the test checks whether you are doing Scrum (in Nokia's opinion):

• •You know who the product owner is
• •There is a product backlog prioritized by business value
• •The product backlog has estimates created by the team
• •The team generates burndown charts and knows their velocity
• •There are no project managers (or anyone else) disrupting the work of the team

My reaction:
I think this is an excellent way to deal with Cowboy Agile or Agilefall.

Let me say this loud and clear: a firm can't in good faith say "we tried Scrum" and then move away from it if they never had any teams that could pass the Nokia test. And pass for a reasonable period of time. Of course they could say "we tried Scrum", but they did not.

The Nokia Test is demanding. That is clear. But a test of this nature is a necessary (if not sufficient) condition to doing professional agile software development. (Or did we expect to get out of Fred Brooks' tar pit by doing unprofessional software development?)

There are some forces in a firm that want to do Cowboy Agile or Agilefall or want Agile to fail ("it's moving my cheese"). This is true in every firm that I have worked in, I believe. So, if you use the Nokia Test, expect to get some resistance.

The Nokia Test is arguably too minimal. There are many other important parts of Agile or Scrum that it does not cover. Are the things in the test the most important parts? My thought is that this collection of principles and practices provides a good core of Agile/Scrum that will defend you from the most common dysfunctions. Not all. Is there a better test? Probably we could define one, but let's pass the Nokia Test now.

Should the test be significantly more detailed? I think not. First, to work with any test, we need the test to be simple enough to be comprehended easily by most of the people involved. In this way, it becomes self-policing. Second, Agile/Scrum needs to be adaptive, so having a lengthy test that puts all teams in the same straight-jacket would noticeably limit that adaptability.

What are your thoughts? Are you aware of similar tests? How does your firm limit Cowboy Agile?