The Essence of Scrum: An Interview with Kenny Rubin
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
You may already know Kenny Rubin—he's one of the agile industry's most prolific trainers and coaches. He has trained many thousands of people on Scrum and agile and has coached a fairly impressive list of clients on these same topics. He's also the first managing director of the worldwide Scrum Alliance and frequently speaks at industry conferences.
Kenny's story begins in the world of object-oriented development, particularly Smalltalk, where Scrum (and agile) originated. By the time the first Scrum project was initiated in 1993, Kenny was already an eight-year object-technology veteran; his first Smalltalk project was an expert system for the Goddard Space Flight Center of the National Aeronautics and Space Administration (NASA), in 1985. After the Goddard project, Kenny went to work at ParcPlace Systems, a startup company formed by Xerox PARC, the famed Palo Alto, California–based arm of Xerox that invented many key early computer technologies, such as graphical user interfaces and the Ethernet. The charter of ParcPlace was to bring object-oriented technology (Smalltalk) out of the research labs and release it to the world. At ParcPlace, Kenny was the manager of the Methodology Development and Professional Services departments.
During the late 1980s and early 1990s, Kenny consulted with a variety of organizations, helping them to use Smalltalk and early agile practices to develop innovative solutions. His first formal use of Scrum was in 2000 at Genomica Corporation, where he and Mike Cohn collaborated on the development of bioinformatics software. After that, Kenny spent his time in the executive suites of seven different startups, holding positions such as CEO, COO, VP of Engineering, and VP of Product Management, as well as a two-year stint at IBM. The companies he worked for have raised a total of $150 million in investments, including two public offerings on the NASDAQ and a series of venture capital rounds.
After his product development roles, Kenny decided to focus on increasing the capability of the industry, first by designing the core model of Agile University for Rally Software; later, by serving as the first Managing Director of the Scrum Alliance. When Kenny took on that role, the organization had 10,000 Certified ScrumMasters (CSMs); today, that number is over 150,000. After fulfilling his one-year commitment to the Scrum Alliance, Kenny focused full-time on agile training and coaching. Along the way, he managed to complete his book Essential Scrum: A Practical Guide to the Most Popular Agile Process, which took the better part of three years to write.
I asked Kenny for a few minutes of his time to explain the "why" of the book—what motivated him to write it, and what the book might mean to the agile development community.
Matt Heusser: Thank you for your time today, Kenny. Let's cut right to the chase: Why did you write this book?
Kenny Rubin: First, let me say thanks for interviewing me!
I wrote the book to fill an important void in the industry. I teach many Scrum classes, and in almost every class I get the question, "Which single Scrum book would you recommend to help teams be successful?" I found that there wasn't a single up-to-date book that I could recommend covering the core Scrum framework and the most popular approaches for applying it. Instead, I rattled off a list of books that I felt would provide the requisite knowledge. That list would add up to several thousand pages of reading, which is overwhelming. So I decided to write the book that would be the single, standalone resource for essential Scrum knowledge.
Matt: Most Scrum books are for people on the development team. Why should a manager or executive read your book? It's 500 pages long!
Kenny: There are really two parts to this question: 1) Why should managers or executives be familiar with Scrum? and 2) Why in particular should they read my book? To start with, there's a belief that agile and Scrum focus only at the development-team level. That's myopic—a road to failure. In my experience, companies that are successful with Scrum have managers and executives who have come to embrace agile principles. So I strongly believe that managers and executives must be familiar with agile principles and Scrum practices.
Now for the second question, "Why should managers and executives read my book?" My original goal was actually to write a book specifically for managers and executives. I was going to call it "Scrum: A Manager's Guide." The book was intended to appeal to managers and executives by taking an economic approach to Scrum and agile, which would distinguish it from many other agile books that take a developer-centric or techno-centric perspective.
As I began to write that book, it became clear that my approach of having a simple-to-read, easy-to-understand, visually rich, economically grounded Scrum book would benefit not only managers and executives, but all members of the Scrum team. So the book morphed into Essential Scrum: A Practical Guide to the Most Popular Agile Process. Basically I inspected and adapted the focus of the book as I started to write it and received feedback from reviewers.
I was comfortable with this pivot because I never bought into the myth that technical people are only swayed by technical arguments and only want to read technical books. In my experience, the best technologists make economically sensible decisions that guide their technical tradeoffs, and therefore they appreciate the style that resonates well with managers and executives. By sharing a common approach, these audiences get a common vocabulary. They're using the same framework for discussing Scrum and its implications.
As for the book being 500 pages, when you extract the front matter, reference materials, and back matter from the book, it's actually only 400 pages of content. Also, there are 208 figures in the book—a continuation of the visually rich approach I was using to appeal to managers and executives. With a powerful, reinforcing figure appearing every other page, on average, the book turns out to be a rather fast and visually appealing read!
Matt: Your book adds new diagrams and graphics to explain what Scrum is, and you create a new sort of "visual icon language" to describe it (see Figure 1). How did that language evolve? I know trainers are using it to describe Scrum—do you see teams or other applications using it to communicate?
)
Figure 1: Image from Essential Scrum: A Practical Guide to the Most Popular Agile Process. Used with permission.
Kenny: The new visual icon language in the book evolved out of work I was doing for my Scrum training classes. I wanted to have an agile/Scrum icon language (a visual vocabulary) for composing powerful, three-dimensional graphical representations of core agile concepts and Scrum roles, artifacts, and activities. Many one-off pictures of these concepts already existed, but none of them appealed to me. Furthermore, the most popular version of the Scrum framework picture (the equivalent to my image in Figure 1) specifically didn't show core Scrum practices (such as sprint review and sprint retrospective). So I decided to create my own visual language. When I began to write Essential Scrum, the creation of the visual language was well underway, but the book accelerated it into high gear.
I freely admit that I'm not an artist, but I am a concept guy. I brought in a fantastic graphics artist with whom I've collaborated for many years, and I asked him to work with me on the new icon vocabulary. The development of the language was very iterative; I would describe the concept of what I was trying to achieve, and the artist would come up with visual ideas for how to represent that concept. Then I would select candidate ideas, and we would iterate designs. As when working with any reusable asset, I needed to keep iterating until we had used each icon in several compositional pictures before I would know whether what we had conceived was useful and reusable. I was leveraging an old adage from my Smalltalk days: "You don't know that something is reusable until you have actually reused it several times. And be prepared to build it at least three times before you have something that's reusable." That's pretty much how it went with the icon-language design.
Figure 1 shows the core Scrum framework. To me, this was the most important of all of the images, because it's the figure that most people want to reference and use. In that one figure you get a beautiful, three-dimensional rendering of the complete Scrum framework. Most of Chapter 2 in Essential Scrum focuses on discussing this picture and its nuances. Rather than try to repeat that discussion here, I recommend checking out that chapter in the book.
As for its broader uses, I've already been approached by many Scrum trainers who want to include this picture (and more generally the visual language) in their own presentations, as well as Scrum practitioners who want to use this picture to communicate Scrum concepts better within their own organizations. I expect that this image will become the new visual standard for representing Scrum. I'll be making the visual language available from my Innolution website.
Matt: In the book, you address topics like technical debt and higher-level planning. Why do you feel these are important in a Scrum book, since many people think of Scrum as a team-level development process?
Kenny: You're right! Many people pigeonhole Scrum as a team-level development process. I believe you must address more than the team level if you want to realize the full benefits of applying Scrum, so I included these topics because I consider them "essential."
Many of the problems in organizations start at the highest levels of planning—such as portfolio-level planning. When these higher levels of planning are poorly aligned with agile principles, you can be certain that the misalignment will manifest itself in an ever-increasing snowball of problems that roll downhill toward the development teams. For example, working on way too many products/projects at one time is a classic problem at the portfolio level, experienced by pretty much every company I see. People at the team level are pulled in so many different directions that they have a hard time staying focused and getting the job done. The concept of managing work in process (WIP) applies as much at the portfolio level as it does at the team level, and I wanted to make this alignment clear.
Remember, my goal was to have a single book that provided essential Scrum knowledge, so I had to address these topics (such as the full gambit of planning topics) if I wanted to tell people in good faith that my book covered what was truly essential to be successful with Scrum. Now they can read one book and get the full end-to-end picture. To me, it was critical to communicate how I—and the companies I work with—perform these activities, and to do so in one place so a person could see all of the planning levels presented as an integrated whole. Other books are dedicated to portfolio planning, release planning, and so on; people can read those books to explore additional material on those topics.
Some people find it curious that I included a chapter on technical debt—and placed it in the section on core concepts. Of course, technical debt is not specific to agile or Scrum, but how to manage technical debt when applying Scrum is a topic of importance to every organization that applies Scrum—since they all have technical debt. I leverage that concept in many chapters throughout the book to ensure balanced economic discussions.
I think book reviewers agree. Many of them have pointed out that my inclusion of these topics and how I wove them together into a cohesive framework are critical aspects of the book.
Matt: The book has a detailed chapter on core "agile" principles. You seem to have gone beyond the traditional Agile Manifesto that so many other books used as a foundation. Why?
Kenny: Good observation. Many agile books describe core agile principles by rehashing the Agile Manifesto. I think the principles in the Agile Manifesto are important, but other principles aren't mentioned specifically or are only implied by the Agile Manifesto, and those other principles really are needed to lay a solid foundation for the application of agile.
I'm not concerned with labels, such as what is or isn't "agile." My goal was to lay out the principles that I considered to be essential if you want to be successful in applying Scrum. I go on in subsequent chapters of the book to show how these core principles are applied.
So what's the short answer to why I cast a net wider than the Agile Manifesto? Because I needed to.
Matt: You've have spent years training and coaching teams and organizations on how to leverage Scrum. In that role, I'm sure you've seen plenty of things that just never work. What are some of the bigger issues you see that frequently cause organizations to fail with Scrum?
Kenny: Scrum is a tool that helps expose the dysfunctions that plague organizations. Some organizations have no appetite for addressing these dysfunctions, and they actually become quite uncomfortable when the process makes those problems visible. These companies are likely to adjust the (Scrum) flashlight to obscure the problems, rather than addressing them head-on. Organizations that are poised to be successful with Scrum are prepared to face the brutal facts head-on and address their impediments.
Let me give you an example. In one of my training classes, I made the following remark: "Programmers and testers should be on the same Scrum team, and their goal should be to produce features by the end of the sprint that have zero known defects." Immediately a lady in the back of the class raised her hand and said, "That won't work here!" I asked why, and she responded, "Well, I'm the manager of QA, and all of the QA people report to me. You just said I should assign all of my people to Scrum teams, and they should help ensure that their teams produce zero defects." I said, "Yes, that's right. I know that's an aggressive target that you might not achieve initially, but why not have it as your long-term target?" She said, "No, that won't work because we compensate our testers based on the number of defects they find, so if they have zero known defects at the end of the sprint, their compensation will actually be less."
To me, the telltale sign of whether this organization could be successful with Scrum hinged on what she said next. If she said, "Well, in that case I'm not going to assign my testers out onto Scrum teams," then that organization would likely fail with agile. When presented with a real impediment (how they're currently compensating testers), rather than face the impediment head-on, they would prefer to redefine Scrum (by not having cross-functional development teams). Actually, in this case the manager said, "I guess I will need to have a conversation with the VP of HR and the VP of Engineering to discuss how we will change the tester incentive-compensation model." That's a very good sign.
Matt: At the same time, I'm sure you saw your share of success. For the organizations that took hold of Scrum and stuck, how were they different? What did they change first? second?
Kenny: First, the organization needs to face the sometimes brutal reality of its impediments and deal with them head-on. That answer is a natural follow-on to the previous question.
Second, there must be significant buy-in from senior management if the organization is to be successful with Scrum. Having a high-performance Scrum team that produces potentially shippable software every sprint, but must operate within an organization where that value cannot be exploited in a timely way, doesn't really provide the expected benefits of agility. Senior management has to align the value chain for success. That's why, in a previous question, I mentioned that having managers and executives up-to-speed and on board with Scrum is critical.
You may find it interesting to know that a frequent comment in my classes is, "You're preaching to the choir. We agree with what you're saying but we can't do anything about it. The people who can do something about it aren't sitting in the room, and they need to hear this." Senior management not only needs to hear it—they need to live it. The good news is that I'm doing more and more executive training these days. And, for executives who are "too busy" to sit in a room for a day or so, they can now read Essential Scrum at their leisure.
Finally, organizations that successfully apply Scrum learn fast. In other words, they cycle efficiently through the learning loop of "assume, build, feedback, inspect, and adapt."
Matt: Opponents of Scrum claim that its focus on iterations tends to cause bottlenecks. At the beginning of the process, the testers have nothing to do; at the end, they're slammed, and the developers are bored. Have you seen this problem? Do you think the solution is to drop iterations entirely and limit work in progress, Kanban style? Why or why not?
Kenny: I frequently hear this claim when I visit companies. To me, the root cause of this problem has nothing to do with iterations or Scrum versus Kanban. This a flow problem. The development team simply hasn't yet learned how to best organize and manage the flow of the work to ensure that it gets done. In many cases, these teams are applying the concept of a mini-Waterfall within their sprints. Think of a mini-Waterfall as analyzing all of the sprint product backlog items first, and then designing all of them, coding them, and finally testing them. If the team organizes its work this way, the developers will be more active at the start of a sprint, and the testers will have a crushing load of test work near the end (and they probably won't get it done).
It's not the concept of an iteration that causes this problem. Instead, it's failing to understand that there can be a more efficient way (than a Waterfall) to sequence work to achieve better flow and get more done. I would suggest that these teams focus on their technical practices and consider important techniques such as test-first development. These techniques will help them to understand that testing doesn't "naturally" occur at the end, but instead testing and development occur in a much more interleaved fashion.
The issue of a work-in-process limit is interesting because of the misperception that Kanban has WIP limits and Scrum doesn't. Of course that isn't true. Every time the team plans a sprint, it's establishing a WIP limit—pulling some of the product backlog into the sprint, instead of pulling all of it in. Also, the team dynamically determines a WIP limit during the sprint by deciding how many of the product backlog items pulled into the sprint it will work on simultaneously. Let's say a team has agreed to work on 10 product backlog items in the current sprint. A team applying a mini-Waterfall approach would likely start working on all 10 product backlog items at the same time—effectively working without a WIP limit. A team that's better at managing its flow would determine the smallest number of product backlog items it could work on simultaneously, and then the team members would swarm on just those items (effectively establishing a WIP limit), applying good technical practices to ensure that no large batches of work built up at any point during the sprint.
Matt: To follow up on that last question, other critics of Scrum (and Kanban as well) say these are all just management fads that will go away over time. How would you respond to that comment? What have you seen in your work that shows this isn't the case? (I'm assuming you think that the critics have it wrong.)
Kenny: I'm an old object-technology guy. I was there when object technology was first commercialized in the mid-1980s. The same criticism was lodged against object technology: "This object stuff is just a fad, and everyone will realize that structured design and development is really the best way to go once this fad passes." Well, 25 years later the fad hasn't passed, and object-oriented development is the foundation for much of software development. We used to joke back then, "People today say that they're going to their object-oriented design meeting; in the future, they'll just say they're going to a design meeting, because the 'object-oriented' part will be implied." I imagine the same will be true with agile software development.
Agile isn't some fad that will just slowly disappear over time; instead, agile represents the right tool for the right job. I'm not saying that phase-based, plan-driven, sequential development (also known as Waterfall) is bad. It's just the wrong tool for most types of product development. I imagine in 10 years people won't even be talking about agile anymore, because it will just be the way that successful organizations develop and support their products. I look forward to that day.
Matt: Thank you for participating in this interview. Where can we go for more information?
Kenny: If you want to learn more about essential Scrum, I suggest you pick up a print or eBook (Kindle) version of Essential Scrum: A Practical Guide to the Most Popular Agile Process. My Innolution website has a host of information regarding essential Scrum, the book, and my blog, and you can go there to learn about and download elements of the visual Scrum language. You can also follow me on Twitter (@krubinagile) and on LinkedIn (kennethrubin).
Thanks for taking the time to interview me, Matt!