Sustainable Software Development: An Agile Perspective

Description

• Copyright 2006
• Dimensions: 7-3/8" x 9-1/4"
• Edition: 1st

• Book
• ISBN-10: 0-321-28608-1
• ISBN-13: 978-0-321-28608-6

"Over the years I have seen the software development pendulum swing from one extreme to the other, as deficiencies in 'best practices' at one end of the spectrum spawned a new set of 'best practices' at the opposite end. Kevin Tate's book has finally brought the pendulum to a screeching halt, right about dead center. This book provides a balanced and practical guide to what's important if your goal is to develop software that lasts."

—Mary Poppendieck, Poppendieck.LLC. Author of "Lean Software Development"

"1) In this very practical and accessible book interspersed with real-world examples and personal opinions, Kevin has distilled his years of developing quality software into a set of principles and practices that have been proven to work. If you are thinking of introducing an agile development environment (ADE) into your organization or of improving the one you already have, this book will help you clearly understand the benefits of a sustainable ADE, establish the practices to make it happen and coach you through the follow-up required to change the culture of your organization to make sure the changes take hold.

I am currently faced with exactly this challenge and this book has already given me several ideas I am looking forward to trying out.

2) In an industry plagued with missed deadlines despite long overtime hours, this book offers a refreshing alternative: a set of guiding principles and simple practices to follow that allow you to get the job done by working smarter, not harder. Drawing on the author's extensive experience developing quality software, the book clearly explains the principles behind a sustainable agile development environment, why it works, the practices to make it happen and the follow through required to turn these practices into habits."

—Peter Schoeler, Technical Director, Artificial Mind & Movement

"It's a familiar scene—the schedule's tight, people are putting in heroic efforts to get everything done, then at the last minute a change request comes in that wipes out the gains you had finally managed to make in meeting your ship date. Looks like it's pizza at your desk for the weekend again! An unfortunate situation to be in but a pattern that repeats itself all too often. "Sustainable Software Development" offers hope to break this cycle. It shows how a change in mindset can free you from the tyranny of unrealistic expectations and brings development realities out onto the table for everyone to see. By following these techniques you will be able to define and manage a software development environment that will work for the long haul."

—Kevin Picott

Software development for immediate success and long-term sustainability

Sustainable Software Development brings together principles and practices for building software that is technically superior, delivers exceptional business value, and can evolve rapidly to reflect any change to your business or technical environment.

Kevin Tate shows how to eliminate practices that make development unsustainable and replaces these practices with a sustainable approach that draws on the best ideas from both agile and conventional development. Tate demonstrates how to balance rapid releases and long-term sustainability, achieving both rich functionality and superior quality. You'll learn how to build a development organization that is more productive and can continually improve its capability to handle complexity and change.

Writing for developers, architects, project leaders, and other software team members, Tate shows how to:

• Take control of your development environment, so you can outship your competitors, leveraging new technologies and responding to new business opportunities

• Maintain a consistent pace that optimally balances short- versus long-term requirements

• Keep your code base in a "near-shippable" state between releases

• Prevent defects, rather than just recognizing and fixing them

• Invest continually and cost-effectively in software design improvements

• Leverage the fundamentals of the craft of software development

• Integrate sustainable processes with Agile and traditional methodologies

© Copyright Pearson Education. All rights reserved.



Extras

Related Articles

How To Develop Software Faster

Methodology Design: The Way We Do Things Around Here



Sample Content

Online Sample Chapters

How to Achieve Sustainable Software Development

Unsustainable Software Development and its Causes

Downloadable Sample Chapter

Download the Sample Chapter related to this title.

Table of Contents

Acknowledgments.

About the Author.

Foreword.

Introduction.

1. Sustainable Software Development.

2. Unsustainable Software Development and Its Causes.

3. The Principles of Sustainable Software Development.

4. Working Product.

5. Defect Prevention.

6. Emphasis on Design.

7. Continual Refinement.

8. Culture Change and Sustainable Development.

Appendix 1: Practice Summary.

Appendix 2: Extreme Programming and Sustainable Software Development.

Appendix 3: Sustainable Software Development and the CMM.

Appendix 4: Recommended Reading.

Conclusion.

References.

Index.

Preface

Sustainable Software Development: An Agile Perspective

Introduction
When I was a computer science undergrad in university, I received a coffee mug for Christmas. You may have seen this mug, it has a number of tongue-in-cheek expressions and quotes about software development and software programmers on it. Here's a random sampling:

• If a program is useless, it will have to be documented.
• If a program is useful, it will have to be changed.
• The value of a program is proportional to the weight of its output.
• Any program will expand to fill any available memory.
• Program complexity grows until it exceeds the capability of the programmer to maintain it.
• Hare's law of large programs: Inside every large program is a small program struggling to get out.
• A carelessly planned project takes three times longer to complete than expected; a carefully planned project will take only twice as long.

You get the idea....My mug is copyright 1980 by Art 101 Ltd., and I find it interesting that most of the sayings on this mug have stood the test of time, apart from one that mentions "job control cards" and an antiquated list of programming languages such as JCL, Algol, and PL/1. However, one saying on this mug still stands out to me today:

Weinberg's Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization.

I freely confess that this particular saying really gets my ire up. Perhaps I'm overly sensitized. I've had people quote it to me on a number of occasions; in particular, a former boss who felt that software developers lacked professionalism (he of course was a professional engineer and had the ring to prove it). Certainly what Weinberg says has a kernel of truth to it—that software developers can be a sloppy lot. But, by stating it as a law crafted as a negative statement, it implies that programmers can't write good software and don't pay attention to architecture, and that builders do. This is a slight to professional programmers, on the one hand, while being overly generous to builders on the other.

I am a big believer in the power of metaphor, especially in the context of technology. Sometimes, when faced with a highly technical term or feature to develop, it is incredibly valuable to use a metaphor that ties the term or feature into the real world through a simple comparison that everyone can understand. This is used in Extreme Programming, not to mention in many other technical fields such as usability analysis.

My real objection to Weinberg's Law is that it implies that a valid metaphor for software development is architectural engineering (i.e., building a building). It is astonishing how many people, from users to managers to software developers, believe this is a valid metaphor for software development when asked. Certainly, it is tempting to think that software is like building a building, and frankly what software developer, team leader, or manager wouldn't want to use the inspiration of Frank Lloyd Wright or the Sears tower in Chicago as a metaphor for his or her work? Well-architected buildings are certainly interesting to look at, and exceptional buildings help define cities.

Figure I-1
These buildings show different architectural styles and through their style and aesthetics also demonstrate the temptation to use buildings as a metaphor for software development. Left image ©Kathy McCallum, right image ©Steve Lovegrove. Images from BigStockPhoto.com.

Why isn't building a building a valid metaphor for software development? The short answer to this question is that buildings are largely static and software is not. Buildings are very expensive to modify after they have been built, and indeed the entire process used to design and construct the building is intended to maximize the chances that the building will meet the customer's requirements and will, within some margin of error, stay within the desired budget. It's easy to identify when a building is complete, but successful software never is. Software is cheap to modify, and any successful piece of software is virtually guaranteed to change over its lifetime. Actually, the software industry has trained its users into believing that major modifications are possible, if not inevitable and expected.

Good software developers realize that their programs can be modified to add just about any unanticipated feature—of course, there are limits. Changing a web browser into a database or a scientific visualization application into a word processor might involve huge rewrites, but the key thing is that good software programs with good developers can do just about anything they want. "It's only software" is a common mantra.

Buildings, on the other hand, are another matter. Ask an architect to make an unanticipated change and chances are she'll reply that the only recourse is to tear the building down and start over. Ask an architect to design a building that will handle all the likely configurations and he'll refuse the project. This is because there are only a limited number of possible changes the architect and customers can make to the building over time. In software, unanticipated features are highly likely because software is developed in a highly complex, hard to understand, technical, and constantly evolving environment. After five years of use and modification, the source code for a successful software program is often completely unrecognizable from its original form, while a successful building after five years is virtually untouched.

These differences alone should be enough to convince you that software is not like "building a building" at all. And yet, the methodology taught in school to software developers, the methodology used in most software companies, and the methodology described in the bulk of today's software literature is exactly the same basic methodology, that is, used in building construction and other forms of engineering; the so-called "waterfall" method, which advocates thoroughly understanding user requirements before designing the software, followed by coding, and then testing.

I don't mean to imply that the waterfall method is plain wrong. Great works of engineering such as buildings, airplanes, bridges, and ships have been built with this method. And many of the practices introduced in the waterfall method are good. The problem as I see it is that too few people question the methods they are taught in school and the workplace. This is a human trait more than anything else and not a reflection on software developers in particular. It's just that probably the majority of projects would be better off using a more appropriate metaphor for software development.

If building a building is not a valid metaphor for software development, then what is? My favorite metaphor for software development is a coral reef. I think good software like Adobe's Photoshop, Oracle, SAP, Linux, and Microsoft Windows creates an entire ecosystem of customers, plug-in developers, distributors, VARs (Value Added Resellers), service and consulting firms, hardware manufacturers, and even competitors. This software ecosystem is like a coral reef.

A successful software product plays the role of the underlying coral in the reef. The coral is what makes the entire ecosystem possible. None of the other organisms can survive without it. And just like software, a coral reef is continually evolving, growing, and changing (though, of course, at a different pace!). It's not just the reef itself that is changing, it's the entire ecosystem with all its dependent fish, invertebrates (sponges, fans, etc.) that change and evolve together. The reef ecosystem is incredibly complex, yet also fragile and vulnerable, just as a software product's ecosystem is prone to disruptive technologies or competition. This level of complexity, growth, constant change, evolution, and vulnerability I believe accurately portrays the complex world that today's software products inhabit.

If you are a scuba diver, you should be able to appreciate this metaphor. Coral ecosystems are truly a wonder. You can view them from a distance or up close, noticing schools of fish, large fans, sponges, and hugely interesting and varied coral formations. And if you are fortunate to have visited the same reef over the course of a few years, you will also know how much a reef changes from year to year and season to season, despite the common misconception that coral is more like rock than a living organism. And the complexity of the ecosystem is obvious because it is impossible to understand the entire ecosystem at almost any scale, both when you can see the entire reef, and when you are watching only a small portion of it.

Figure I-2
Nature provides many useful metaphors for software and software architecture that accurately reflect the nature of software development. Successful software is like a coral reef because it inhabits a complex and continually evolving ecosystem of both complementary and competing technologies and software. Image ©Jason Sah. Image from BigStockPhoto.com.

Why does the metaphor we use to think about software development matter? Metaphors help us analyze a problem from a unique and common perspective. Good metaphors help groups of people grasp a complex concept or problem easily and clearly to understand what they are trying to build. In the software industry, we have been using the wrong metaphor, largely unconsciously, and have adopted practices from engineering that reinforce this metaphor.

Too many development teams develop software products as if they were building a building. They place too much emphasis on features of the building and trying to completely understand its requirements before it is built. The temptation is to finish the project or product and then to stand back and admire the creation. Inevitably, however, unanticipated changes come along in the form of user requests, bug fixes, and changes to other software or hardware that the program is dependent upon. The result is that over time, doors get added in odd places, walls get thinner and are moved around, cables get run through elevator shafts, and holes get added between floors or rooms. A mess accumulates that becomes increasingly hard to manage, and the greater the mess, the easier the temptation to add to it. Add in competitive pressures and a constantly changing ecosystem of underlying technology (operating systems, web browsers, databases, UI toolkits, etc.) and chaos ensues.

These projects are on the path to unsustainability. At some point the whole edifice will come tumbling down and become unmanageable. One of the main features will be an increasingly high cost of change where every change, no matter how minor, introduces a risk that something completely unrelated will break. The programmers will blame management, and management will blame everyone else. If the company can manage it, the application will be rewritten, only to start the clock ticking on the unsustainability cycle again. These projects aren't any fun, yet they are distressingly common in the software industry. The software industry is not slow-paced after all; laggards do not have high odds of survival.

This book, as you've gathered from the title, is about sustainable software development. The coral reef metaphor is an important one because the software industry needs to recognize that its principles and practices must support free-flowing, adaptive, and opportunistic evolution with the goal of surviving in the short term and thriving in the long term. Software projects are typically approached to meet one of short or long term, but not both. Either the project is a fragile edifice where the capability to meet customer needs over the long term is in doubt, or the project team spends endless analysis time that compromises its ability to ship, if indeed it ever happens. To deal with the short- versus long-term paradox, we need to embrace change, not be afraid of it, and we must recognize that software is complex, not pretend that it is simple. No software product stands completely on its own, and the development team must be able to interact with and foster its ecosystem. Good software products, companies, and the open source movement clearly exhibit these traits.

The principles and practices we use for project management, software architecture, and even those that help us effectively work together are hugely important in defining the richness of the ecosystem we want our product to inhabit and/or create. Linux, for example, would not be as successful as it is today if it weren't managed as an open source project, didn't have a modular architecture that allows multiple programmers to modify it at once, and have a license that allows businesses to freely use it.

Sustainable development is a pace that can be maintained indefinitely. This does not mean slower development—quite the opposite. Sustainability is not attained by artificially limiting work weeks to 40 hours as some books claim; the number of hours worked is an outcome of being in control of the project and having software with a consistently low cost of change. Sustainable development requires a singular focus on a form of technical excellence that regularly provides useful software to customers while keeping the cost of change low. Achieving this is not easy because the more common forms of technical excellence concentrate on designing and writing perfect software while being change adverse.

This book describes the principles and practices required for change-embracing technical excellence; these principles and practices promote sustainability and result in faster development with less effort through having a consistently low cost of change. For teams who are used to unsustainable development, the experience is best characterized as liberating because they are able to deal with change, not be afraid of it or view it as a risk.

Sustainable development and technical excellence are vital parts of agile development. The agile development movement is challenging the traditional approaches to software development. As a result, I believe the software industry is on a path to creating a widely accepted new way to develop good software that uses the best practices from both traditional and agile practices. However, this is going to take time because agile development is so new and is only now starting to be noticeably employed in a widespread manner. It's going to take time for the mindset and practices to sink in to the industry, to be accepted, and to eventually be incorporated into the education given to new software developers. In short, I think the current debates in the software industry are healthy.

What I find particularly enlightening (and heartening) is that software developers aren't the only ones challenging the standard waterfall development practices. Huge and successful manufacturing companies are using lean manufacturing practices to get their products to market faster, and even traditional construction companies are beginning to embrace lean construction as a way to reduce costs and increase efficiency. Underlying all these new approaches, even though they are completely different industries, is the need to connect with customers to ensure that the resulting work meets the needs of customers, is built in the least possible amount of time with optimal efficiency, and essentially embraces the chaos of the short term to maximize the chances of long-term survival.

Balancing the short and long term requires projects to be more driven by a vision than tracked against a plan. Just as important, though harder to describe, is the conscious tradeoff being made in these approaches toward good enough analysis and documentation of the results of the analysis as opposed to an emphasis on the documents as the guarantor of success. By its nature, this relies on the people doing the work to be self-disciplined and professional as opposed to thinking that a process with its mandatory documentation applied to that group will provide the necessary results. This is a roundabout way of stating that agile development and lean methods emphasize flexibility and collaboration over stifling bureaucracy to minimize wasted effort. And this is why agile and lean practices are as much about culture change as they are about practices.

Where This Book Fits

I wanted to write this book because I see a need for it. Quite a bit of the agile development literature focuses on project management practices that promote agility. Project management practices are, of course, vital. But just as vital is the realization that in order to be agile, your software itself and the methods you use to develop it had better support your ability to be agile. You can't be agile if your software is brittle and is difficult to change with too few safeguards in place to enhance confidence in the changes. Software project management and development practices and software must go hand in hand.

I also feel that too many software projects focus exclusively on features and bug fixing and not enough on excellence. Software quality is an issue for our industry, but we too easily accept it as inevitable and somehow a result of the need for constant change. But as software becomes increasingly important in our daily lives, issues like security and quality are becoming more urgent. I think it's time we find ways to achieve features AND quality, not just view them as exclusive to each other.

As I see it, this book falls into a bit of an awkward spot. While this book is heavily influenced by the agile development community, and Extreme Programming (XP) in particular, agile in my opinion leaves too much to the imagination of the reader/listener, and this leads to many misconceptions. Some of these misconceptions are dangerous and very false—for example, that in XP there is no software design and that teams who practice agile development are somehow less disciplined. What I have tried to do in this book is expand on the existing agile literature by drawing on my work experience to introduce a team-oriented approach to achieving short-term and lasting excellence on a software project through principles (mindset) and practices.

I don't mean to imply that the outcomes, principles, and practices I describe in this book aren't employed at all in agile teams. I know they are, but sporadically and inconsistently. By not talking about these practices, it is too easy for traditionalists to dismiss agile development because it does not openly address some critical practices. The opposite is also true: Many of the practices in this book are advocated by the traditional software development community and are thus dismissed by some agilists. But I think this misses the point.

What I try to emphasize throughout this book is that what separates agile from non-agile practices is the perspective to the practice, not the practice itself. Hence this book's subtitle: An Agile Perspective. An agile perspective means in a nonbureaucratic way that emphasizes trying new things out and always learning, changing, and adapting as circumstances dictate—and with lots of collaboration. So maybe this book is a little bit like a bridge between the traditional and agile camps, but with an unerring bias toward agility. I hope that the saner minds in the agile community agree with me.

The Three Foundations of Agile Development
Agile development practices have three foundations: project management, technical excellence, and collaboration. Using the flywheel analogy from the excellent book Good to Great Collins 2001, agility is just like trying to spin a massive flywheel. The forces that can be applied to the flywheel are the principles and practices you use for software development, which can be categorized by agile's three foundations as shown in Figure I-3. If any of your principles or practices are not agile, then this will act as a brake on the flywheel. That is, appropriate project management practices promote agility, as does an emphasis on collaboration within project teams and with customers, as does an emphasis on technical excellence, and all of these practices combined support agility. If a team employs agile project management practices but non-agile/inadequate technical excellence practices or little or no collaboration, the team's ability to be agile is going to be constrained.

About the Structure of This Book
What is needed for agility is a balanced blend of principles and practices that reinforce each other and the three foundations of agility shown in Figure I-3. The principles and mindset of sustainability are vital because they are the filter through which all the practices are chosen and implemented. Because of this, the core of the book (Chapters 4–7) consists of a chapter for each principle of sustainable software development, with practices that help implement the principle described in enough detail to get started. Chapters 1 and 2 describe sustainable and unsustainable software development, and Chapter 3 focuses on the important distinction between principles and practices. The closing chapter, Chapter 8, is targeted at creating culture change, because change is usually required to achieve a culture oriented around sustainability.

Figure I-3
The three foundations of agile development are the principles and practices used for project management, technical excellence, and collaboration. If agility is considered to be like attempting to spin a massive flywheel, then the principles and practices used for software development either act as a positive force or a brake on the flywheel. Hence, the principles and practices used for all three foundations must support and enhance each other.

Quite a few of the practices described in this book are outlined in greater detail in other sources, and so I have included Appendix 4 with my recommendations for further reading. In general, when it comes to the foundations of agile development, project management and collaboration practices are typically described extremely well in the literature while technical excellence is often left as an exercise to the reader (except of course in Extreme Programming). Hence, I have tried to provide a slight bias toward the technical excellence practices.

I've consciously tried to ensure that this book is not just another technical book. I've encountered too many software developers who are brilliant architects and coders but don't, for example, understand the interdependence between their decisions and their company's larger business interests. I've seen too many projects that redesign their software simply for the sake of an elegant design when a pragmatic approach ("if it works, don't change it") is often in the better long-term interests of the project. I also have seen many teams who focus on creating the best possible software or, even worse, spending an endless amount of time writing requirements and design documents often with NO software and losing their focus on what every project's sole aim should be: shipping!

This book is for software developers, managers, and users who believe in self-discipline, professionalism, and in shipping software products with high quality that last. Especially for those who are looking for ways to embrace change and increase efficiency, while continually improving themselves, their teams, and their organizations.

Foreword

Download the Foreword file related to this title.

Index

Download the Index file related to this title.



Updates

Submit Errata

