Practices for Scaling Lean & Agile Development: Inspect & Adapt

This chapter is from the book 

Chapter

• Thinking about Adoption & Improvement
• Early Days: Team & Management Changes
• Early Days: Breaking Barriers & Habits
• Early Days: Gatherings
• Coaching & Community
• Continuous Improvement

• The taxpayers are sending congressmen on expensive trips abroad. It might be worth it—except they keep coming back.
• —Will Rogers

We have worked closely in a few enterprise-wide lean or agile adoptions over the years, and have collected experiments. Some, covered later in the Continuous Improvement section, focus on scaling and multiteam coordination (such as a Joint Retrospective); many others focus on organizational design and culture. First, a story...

We were coaching in Europe and met with a manager who had been assigned the agile transformation responsibility; he wanted to show us his plan and ask for feedback. He presented a Gantt chart of his planned transformation: many stages of precise duration all in sequence, milestones, specific managers assigned to tasks along the way, cost estimates, and more. According to the plan, in twenty-seven months the group would have transformed to 'agile.' The detail was impressive—it was also the wrong approach.

Our colleague had confused doing agile and being agile. And he was applying command-and-control management thinking combined with predictive planning—in essence, traditional management 'agile' adoption. Fortunately, within a few minutes of chatting, the plan was jettisoned and his view shifted to serving the teams, using a backlog, and adaptive planning.

This misunderstanding to agile or lean adoption is common in corporations that (1) mandate a top-down 'transformation,' (2) think this is another change project with an end ("we have now finished changing to lean—you get the bonus"), or (3) have a centralized group responsible for pushing processes.

Thinking about Adoption & Improvement

Avoid...Adoption with top-down management support

• At a time when all of us are struggling to implement lean production and lean management, often with complex programs on an organization-wide basis, it is helpful to learn that the creators of lean had no grand plan and no company-wide program to install it. [SF09]

"Our agile adoption would be so much better if only we had management support." We have heard that many times, but be careful what you wish for—you might get it! In one enterprise that got official "everyone do agile" management support after an informal adoption had been going on for several years, we hear the complaint, "I wish we never had management support; now people are doing things for the wrong reasons."

Why? In some organizations the culture is

• management phones in their support but does not deeply learn lean thinking or agile principles¹
• management 'drives' change by setting targets and offering bonuses; in this case, the agile adoption targets
• management directs a centralized process group to "push out the new process"

Then, what happens is a superficial cargo-cult agile and lean adoption, with widespread game playing, resentment, hidden resistance, and misunderstanding... another management fad that will pass away if ignored long enough. Perhaps there is a target: "50% of the teams have a ScrumMaster within the year," and managers get a bonus if that is 'true.' Then, existing project or line managers are relabeled as ScrumMasters. Or, "Every product should have a Product Backlog." The existing work breakdown structure of tasks is copied into a spreadsheet called the backlog. Nothing has really changed, and indeed things may be getting worse because of more disruption and gaming.

Avoid forcing—When coaching we encourage: volunteering; do not force any agile or lean approach onto people; people should be left the choice to think and experiment. Create a culture of coaching for those that want to experiment.

Focus—Strive to achieve skill and demonstrate excellence in the adopting groups, with concentrated long-term, high-quality support. The best, most sticky adoptions we have seen had this approach.

Try...Adoption with top-down management support

In contrast to the prior case, we have also seen groups with a high-quality management culture that cultivated genuine improvement.

We recall one client (at a bank) where the leadership team quickly dove deep into reading many books on agile principles, studied and applied systems thinking, all attended a ScrumMaster training with their team members, talked with hands-on experts, used agile coaches, and applied Go See. Quickly after starting, this group had made deep changes in organizational design and there was tangible improvement in the flow of value to users.

For ScrumMasters and other coaches the implication is: Only lobby for top-down support when you think the leadership team is seriously interested in learning and in organizational redesign.

Try...Individuals & interactions over processes & tools

One of our colleagues in an agile-coaching group observed, "This company has tried to use processes to compensate for a lack of competence of its employees."

The first agile value, and the previous story about the effective agile adoption at a bank, reminds us of its veracity—people, not processes, are the first-order effect for success [Cockburn99].² A group cannot 'process' its way out of a deep hole dug by problems with the individuals in engineering and management—'agile' will solve nothing in that case.

So, focus on cultivating and hiring extraordinarily talented people.

But, no false dichotomy... as object-pioneer Grady Booch wrote:

• People are more important than any process. ... Good people with good process will outperform good people with no process every time. [Booch96]

Try...Job and personal safety (not role safety)

• It is difficult to get a man to understand something when his job depends on not understanding it.—Upton Sinclair

We were in Norway, dining on lutefisk with a colleague. He said, "My company has hired consultants for a lean initiative. They are identifying redundant employees and firing them."

This is a perversion of lean thinking. Lean has nothing to do with terminating 'redundant' employees, nor with lean-by-consultants. The English name 'lean' was not chosen to imply removing the fat from an organization. Rather, it was chosen³ to contrast mass manufacturing with lean manufacturing—working in small batches and with less effort to produce goods.

Toyota strives to provide long-term job safety. This is part of the first pillar of lean thinking: Respect for People. And it is also intimately connected to the second pillar: Continuous Improvement. Who is going to strive for continuous improvement in the organization when the likely outcome is job termination? Yet, this does not imply role safety—which inhibits improving the system. For example, project-manager role disappears in Scrum; we have seen people then shift to hands-on engineering or product management.

Personal safety—In Los Angeles one December morning we waited in a room to meet with a team we had been invited (by higher-level managers) to coach for a few weeks. Soon they showed up. We welcomed them and asked, "What are the problems you'd most like to work on? Maybe we can help a little." There was a long silence—people were uncomfortable to openly discuss problems. So, below the extreme case of job loss, there is the issue of personal safety and improvement. In the Crystal Clear agile method, it is identified as one of seven key properties set up by the best teams:

• Personal Safety is important because with it, the team can discover and repair their weaknesses. Without it, they won't speak up, and the weaknesses will continue to damage the team. [Cockburn04]

A book we sometimes suggest to ScrumMasters (and others) is The Five Dysfunctions of a Team [Lencioni02]. The first two of these dysfunctions are absence of trust and fear of conflict. An improving Scrum team must resolve this. See the recommended readings for material that might help.

Offshoring is another context that we regularly see personal safety problems; a company terminates employees in higher-cost regions and shifts more work offshore. This impacts motivation and collaboration between people in different regions.

In a new large-scale Scrum adoption initiative, ScrumMasters and others need to be mindful of these dynamics: Is Scrum or lean development going to be viewed as a mechanism to 'streamline' and terminate overhead? And whereas in little companies active opposition to the system is common, in large product companies there is often a sense of disempowerment and reduced personal safety to challenge the existing organization. Then, for instance, people complain that Scrum Retrospectives are ritualistic, useless, or dead. Or perhaps even worse, people develop a passive-aggressive attitude in response to this 'streamlining,' with subtle negative consequences.

It takes active ongoing encouragement from the leadership to keep kaizen mindset alive. As Toyota CEO Katsuaki Watanabe said:

• The root of the Toyota Way is to be dissatisfied with the status quo; you have to ask constantly, "Why are we doing this?" [SR07]

Try...Patience

Toyota has taken decades to cultivate a lean culture; similar patience is needed elsewhere. Further, Toyota rapidly expanded in the 1990s and then experienced more difficulty in spreading and sustaining a lean-thinking culture, especially in their satellite plants. It is easy to start losing that culture without ongoing constancy of purpose by lean-thinking manager-teachers [Womack09].

Daily stand-ups and visual management can be installed in days. But it takes years to a develop an enterprise of people that know, teach, and apply agile and lean thinking. It is worth it—there lies the great leverage for sustained improvement. Hence the Toyota message, build people, then build products.

Avoid...Adopting "do agile/lean"

Be agile rather than do agile was the theme of the Agile chapter in the companion book. Agile is not a practice; it is a set of values and principles. Some of the clients we work with misunderstand this and establish a large-scale transformation project that is measured in terms of observed practices, such as,

having a Product Backlog	doing a daily stand-up	working in time-boxed iterations
having information radiators on walls	doing planning poker	writing user stories

To be clear, we recommend trying these practices—indeed, the next suggestion emphasizes that doing concrete agile or lean practices is very important. But there is a big difference between a genuine jelled self-managing team that wants to hold a daily stand-up so that they can coordinate, and a group that has been told to have a Daily Scrum—especially if that is on someone's checklist of "practices in place that prove we are doing agile."

It is common to find groups where all these practices are observed, but where there is only superficial adoption or understanding and little or no agility.

Similarly, we recently visited a large outsourcing client in India that was "doing lean." We asked what that meant. Answer: Using a software tool to measure their WIP levels, and trying to reduce it.

Avoid...Being agile/lean without agile/lean practices/tools

"We understand the Agile Manifesto and lean thinking, and focus on the big ideas—we understand that all practices are just context dependent. And the standard tools don't work in our context, because we're different. We have very lean analyst teams, component teams, and test teams, each focusing on their flow."

In addition to seeing shallow practice adoption, we have seen the opposite: A claim to follow agile or lean thinking but no (or little) application of any concrete tools and practices. This is associated with relabeling existing ways of working as agile/lean, when in fact very little has changed or improved.

What happens if there is genuine effort to adopt many agile or lean practices or tools? For example, test-driven development, visual management of WIP (perhaps combined with a limited-WIP policy), reduction in handoff, and more? This doing creates a concrete framework for learning and kaizen, and a force for deep transformation. Without that concreteness, it is easy to (1) miss subtle insights and context-dependent lessons, (2) miss discovering benefits of these tools, and (3) avoid really improving.

Walk before running

In Agile Software Development, Alistair Cockburn [Cockburn07] discussed the shu, ha, ri model of stages of skill development in Aikido and its applicability to practices-versus-principles in agile adoption. This parallels the apprentice, journeyman, master model. People need to walk before they can run—they cannot become masters without first spending time with tools, mastering them by the book, and experiencing different contexts.

The kaizen cycle starts with learning and applying a standard practice⁴ for similar reasons and because improvement should be against a baseline of insight gained by practice. And there is similar advice in Scrum...

• Rule changes should only be entertained if and only if the ScrumMaster is convinced that the Team and everyone involved understands how Scrum works in enough depth that they will be skillful and mindful in changing the rules. [Schwaber04]

Avoid...Agile/lean transformations or change projects

Framing the adoption of lean thinking or agile principles as a transformation or change project leads to the notion

• it is a project, with an end

  • rather than lifelong continuous improvement based on experimentation and growing problem-solving skills

• it is something that people do

  • rather than a change in mindset, culture, and paradigm
• it is something to define and direct by managers

So, rather than framing this as "the agile change project," experiment with framing it as...

Try...Agile/lean adoption forever

One of the pillars of lean thinking is continuous improvement; lean adoption is not a project with an end. Similarly, a group has never finished adopting Scrum; the framework implies inspect-and-adapt every iteration, without stop. Therefore, do not establish an agile change project; rather, build a permanent system for improvement.⁵ And rather than framing what managers do as managing "the agile change project," experiment with framing what they do as...

Try...Impediments service rather than change management

Sometimes, phrases are influential. Consider the difference between manage the agile transformation and impediments service.

In the latter case, in the lifelong agile or lean journey (it is not a project), the team members and Product Owner create an impediments backlog of their impediments—policies, structures, environment, tools, and more. The role of managers—in the context of agile adoption—is to help the teams and Product Owner by never-ending impediments service—working to remove impediments forever.

This change in behavior—and phrasing—is a shift from top-down or command-and-control to bottom-up service.

It leads to more Go See behavior by managers and the chance to serve as teachers rather than controllers or planners. For example, many team members will not even realize something is an organizational design impediment; lean-thinking manager-teachers have an opportunity to help them learn to see this.

Iterative and adaptive; pull from the backlog—This is also a shift from predictive to adaptive planning. In this model, agile adoption is based on (1) a prioritized impediments backlog, (2) short impediment-service cycles⁶ executed by managers, and (3) adaptive iterative planning based on a re-prioritized backlog each cycle. Who knows what will be done in the next impediments-service cycle?—As with Scrum, the impediments backlog is emergent and continually re-prioritized.

Prioritization and impediments backlog owner?—An official backlog owner is probably not needed. Instead, experiment with this: Every team, when they add an impediment to the backlog, give it a priority. Then, prioritize based on (1) number of teams that raise the same impediment, and (2) average priority of the impediment.

Avoid 'impediments' added from quality and management areas—Some years ago, in China, we were coaching a Scrum-adopting product group that had an impediments backlog. All the original impediments came from hands-on workers. But after some time, quality managers and department managers started to add their own 'impediments.' These were not impediments of flow of value to customers, nor impediments from the value-worker viewpoint; rather, they were 'impediments' such as "not conforming to centralized process practice <X>." Avoid that; the important work is the value stream of the teams and Product Owner, and removing their impediments. All that said,...

Avoid 'impediments' added from hands-on workers—If you ask a typical existing team of testers or a component team, "What is the best team structure?" They will say, "Our current structure, of course!" It is common that people—arguably even more so in non-management positions—have not developed systems-thinking or lean-thinking skills, nor have they studied organizational design, team, or product-development research. Toyota (and management thought leaders) have emphasized the vital role of managers who have this kind of knowledge, educate others, and improve the system with insight. Suppose there was a recent shift to feature teams and early testing, and then ex-test-team members added an 'impediment' to the backlog: "the testers should be in a separate group, and avoid testing early so that it can be done efficiently at the end." ScrumMasters and manager-teachers have a responsibility to debug these local-optimization thinking mistakes, and clarify problems that genuinely impede the flow of value. It is easy to fall into the trap of local suboptimization thinking—watching the runner rather than following the baton, forgetting gemba and Go See. We make this mistake too. Testing our ideas against people educated in systems thinking can help.

Managers add system impediments—Building on this last point, there are system weaknesses to the value stream (usually in policies and organizational structure) that team members are unlikely to grasp or consider candidates for change. Managers have a pivotal role in identifying and removing these. The Organization chapter in the companion book centered on these weaknesses.

Add impediments from the Product Owner and product management—The value stream is within the teams and in the work of the Product Owner and product management. Invite product management to impediments backlog workshops.

Accept the priority given by the hands-on workers—At one of our clients in Greece, we facilitated an initial impediments backlog creation workshop with team members. After all the voting, what was their number one impediment?—A slow network. For years that had been the dominant issue (it inhibited integration, for instance), but no one in management had done anything about it—the priority of this and other impediments had never been this clear. Now, with a list of 50 prioritized impediments, the number one issue was unambiguous. To their credit, the management team—that had agreed to move to the model of impediments service—accepted its priority and within a few months, problem solved. This also built trust and cooperation because the teams saw managers genuinely helping solve their key problems.

Create the initial impediments backlog in a workshop—We have helped set up many impediment services for management teams, and have found the following approach useful to start it off:

1. Convene a workshop with hands-on people from teams, the Product Owner, and other product managers. In other words, focus on gemba—where the value work is. Start with brain-writing impediments on cards, in pairs.
   

   Click to view larger image

2. Next, form larger groups from four or five pairs. The groups discuss, merge, and refine the impediments into a new set of cards. Use the floor.
   

   Click to view larger image

3. Combine the refined cards from all groups into a central floor area. Do affinity clustering to group them. Remove duplicates. Then, do dot voting by all participants. Finally, lay out all the cards in (dot voted) priority order. Discuss and refine—final tweaking.
   

   Click to view larger image

4. Use visual management. Set up the backlog on a wall outside the office of a senior manager. (This photo shows a dayone backlog with no 'service' yet). For example, in Greece it was set up near the office of the head of the development center. During impediments-service Sprint Planning, or at other times, managers volunteer for an impediment, write their name on the card, and move it to the middle WIP column.
   

   Click to view larger image

Try...Human infection

Thinking and acting outside the box is possible but hard when everyone is inside it. Lean thinking, agile principles, self-organizing teams, test-driven development, feature teams, manager-teachers... these are mindset, culture, and behavior changes—and to be sticky or meaningful, these kinds of changes require human infection from experts through long-term face-to-face coaching.

In the most successful adoptions we have seen, the organization established internal coaches supplemented with external coaches (both were important), and emphasized lots of hands-on mentoring from these agents-of-change during the real work.

Avoid...Agile/lean adoption targets or rewards

Rewards work. An economist wrote in his blog a story to prove this: His son still wore diapers to bed each night. The economist told his son, "If you don't wet your diapers tonight, tomorrow I'll buy you the toy you want." The next morning, the father went into his son's room. His son had successfully fulfilled the goal and was looking forward to the reward. He had removed his diaper the previous night. The bed was all wet, but his diaper was dry.

The Organization chapter of the companion book summarized the hard facts that performance-based incentives lead to gaming, opacity, and a weakening of the system. We have seen their deleterious effect in promoting "fake agile" adoptions in several groups. Avoid that—and avoid "agile adoption" target setting. The quality guru W. Edward Deming, in his 14 points for management [Deming82], summarized this in number...

• 11. Eliminate management by objective. Eliminate management by numbers, numerical goals. Substitute leadership.

Avoid...Competitive 'improvement'

At some clients we have worked with, the introduction of kaizen gets mixed up with their prior management culture, such as competitive incentives. Then, teams or individuals are offered rewards if they improve more than others. This leads to a competitive rather than cooperative culture, in which parties are less willing to share or help others since they might 'lose' individually.

Avoid...Try...'Easy' agile or lean adoption

'Easy' agile adoption is an existing weak organizational design not meaningfully changed, and a thin veneer of practices painted on: managers relabeled as ScrumMasters, existing component/analyst/testing teams get their own "Product Backlog" and hold a daily stand-up meeting every week, and more. There is no significant improvement, and people do not take continuous improvement seriously—or worse, they think, "the agile adoption is finished."

On the other hand, Scrum emphasizes the art of the possible. It may be that minor modifications are the current limits of change because of limits in mindset.⁷ These will not meaningfully enhance the value flow to customers, but perhaps (1) adding prioritized backlogs, (2) working in short timeboxes, (3) lowering WIP, (4) holding standups, and (5) reducing multi-tasking will help fractionally. It is a first step before deeper change and improvement. Then, we suggest...

• If you're going through hell, keep going.—Winston Churchill

Try...Experiment rather than improve

The mandate to improve is a lofty goal, and can scare off people from experimenting. What if the improvement...doesn't? Kaneyoshi Kusunoki, a student of Taiichi Ohno and executive vice-president at Toyota, said about kaizen and management support:

• A defining characteristic of the corporate culture at Toyota is that managers don't scold you for taking initiative, for taking a chance and screwing up. Rather, they'll scold you for not trying something new, for not taking a chance. Leaders aren't there to judge. They're there to encourage people. That's what I've always tried to do. Trial and error is what it's all about! [SF09]

Developing problem-solving skills through many experiments is central to lean thinking. The only bad experiment is the one not tried!

• The real measure of success is the number of experiments that can be crowded into 24 hours.—Thomas Edison

In this light, the Try... and Avoid... ideas in this and the companion book are just experiments—and also because systems are too complex and variable to assume prescriptive advice will work.

Try...Encourage experiments; offer coaching

The mandate "adopt agile development" is daunting and large. The mandate "do continuous integration" reflects command-and-control, forcing practices. An alternative to both these approaches is to foster the kaizen mindset encouraged in lean thinking: People are encouraged to experiment and are supported with coaching and education. For example, a ScrumMaster can explore with teams the problems associated with delayed integration, describe continuous integration as an alternative, and arrange coaching if the teams want to try it.

Avoid...Adopting <X> because "agile didn't work here"

Survey decades of management and product-development trends, and some patterns emerge. Possibly the dominant one is

1. difficulties exist due to system weaknesses in organizational design, poor engineering skill, and ineffective management
2. try new 'thing' to address a problem (insert: MDD, PMI certification, Kanban, CMMI, Scrum, SOA, agile, next-generation lean, ...)
3. do not address the systemic issues; try 'thing' superficially
4. after two years, abandon 'thing' because "it doesn't work here"
5. go to step 2

We see this in some groups trying Scrum. Scrum is a simple framework that acts as a mirror: Rather than fixing problems, it increases visibility of systemic weaknesses, inviting inspect-and-adapt with experiments. In some groups, rather than fixing the system, it is easier to try the next thing... "Let's call in new consultants specializing in Scrum failure, and then adopt...next-generation lean."

Avoid...IBM/Accenture/... agile adoption

This is not about IBM or Accenture per se; it is about

• the misconception that agile is a process or practice
• shifting responsibility for agile/lean success to an external consulting group

From this stems the notion it can be bought and installed—and there are companies happy to take your money claiming so. Plus, it is related to the misunderstandings summarized in the False Dichotomies chapter of the companion book: agile means iterative development, Scrum means daily stand-ups, and so forth.

Avoid...Adopting agile with "agile management" tools

"We're starting to do agile. What tool should we buy for agile project management?" This is a question we hear often; our suggestion is always the same—and we mean this even for the very large-scale cases: "Avoid any special agile tools until several years after starting the adoption. Keep it simple. Use the wall or, in the most complex solution, a simple spreadsheet and wiki." Why?

Problems from system weakness cannot be solved with processes or tools. Worse, attempting to quick fix systemic problems with tools creates an illusion of control or change but no real improvement... Executive: "What is the agile transformation progress?" Agile-change manager: "We have installed <AgileToolX> and three of the projects are using it. Come take a look at the burndown charts..."

Avoid the lure of "tools to do agile management" for at least several years after starting to adopt agile or lean development, so that people's focus can be where it belongs: on the system. By removing all crutches and quick-fix illusory solutions, people may—just possibly—be prompted to squarely face the important but hard issues: competent individuals, interactions, organizational design, the illusion of command-and-control, and so on.

• If you automate a mess, you will get an automated mess.—anon

We are not suggesting agile-management tools are poor—or good. This is about focusing on important things first and avoiding the dysfunctions that accompany management-reporting tools.

After <N> years? Prefer free tools so that the cost of experimenting is low and there are fewer barriers to discarding tools. We have heard the following several times: "We can't stop using tool (or process) X because we have invested so much in it."

We have seen thousand-person multisite development groups successfully apply large-scale Scrum with some Excel spreadsheets for their Product Backlog and Release Burndown chart. Indeed, they are almost certainly better off for doing so; it keeps their attention more on fixing the system.

Also, there is a more subtle, pernicious danger with agile-management tools. These are the fifth and eleventh agile principles:

• 5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
• 11. The best architectures, requirements, and designs emerge from self-organizing teams.

A theme in Scrum (and other agile methods) is self-managing teams, as covered in the Teams chapter in the companion book. And the fifth principle emphasizes trust and support, which is quite different from monitoring people's work. So what?

The agile-management tools we have seen emphasize tracking and displaying individual and team tasks and Sprint Backlogs to managers—almost the antithesis of these principles. In Scrum, the team's tasks (the Sprint Backlog) are created by the team to help them self-manage, not to report their status to others. As the well-known team researcher, Richard Hackman, explains, "In self-managing teams, the responsibility of tracking the progress is delegated towards the team" [Hackman02]. Since the team is self-managing, they are not to be tracked or monitored; such tools are a slippery slope that may reinforce a traditional command-and-control culture rather than a culture of self-management.

We know a coach who works for an 'agile' tool vendor. He told us that they had been joking about adding a "real Scrum" button to their tool. This button would turn off all the non-Scrum and unnecessary features that were requested by their traditional-management clients...and there would be almost nothing left in the tool.

There is a well-known case of a company where project managers inspected daily the Sprint Burndown charts of teams, and "solved the problem" when the charts did not go down. Ken Schwaber—the Scrum co-creator—was visiting and noticed that all the burndown charts had almost no deviation between the burndown and ideal lines. Eventually he discovered that a team kept two charts: a fake one for the managers so that they would stop interfering, and a real one to support self-management.

Computerized management-reporting tools can also take people away from gemba and the practice of Go See. Lean thinking emphasizes—to understand what is really happening—go with your feet and see with your eyes at the real place of work, help solve problems there, and build relationships with the workers there.

Finally, these tools are optimized for reporting—not for success, improvement, or a better flow of value. What meaningful problem do they solve?