This chapter is from the book
This chapter is from the book
This chapter is from the book
Summary
Your job as a project manager is not to prevent change but to help your stakeholders navigate through the inevitable tradeoffs between scope, schedule, and resources. Avoid assuming the role of the "scope police" and become your client's trusted partner, educating stakeholders so that they can make intelligent choices. The inevitability of change may cause you to throw up your hands in despair, but keep in mind that you are managing a creative process in a dynamic communication medium, not building a suspension bridge. Attitudes and methodologies inherited from civil engineering, the military, and other project management cultures may not fit the open-ended, creative world of Web site design.
Although they represent an exciting development, iterative models are not a panacea to the problem of scope change. Many prototype-centric methodologies require unique conditions for success. For example, rapid prototyping requires a client who is knowledgeable about the Web, responsive, creative, and willing to participate actively in the ongoing design process. Such clients are a very rare breed! The techniques of pair programming advocated by XP enthusiasts often require significant changes in organizational culture and work habits, as well as a suitable personality! While it is true that many new appraoches promise to be better suited to the Web, it is advisable to first acquire a firm foundation in commonly accepted practices. Once you have mastered the fundamentals of scope management, don't be afraid to experiment on small projects. As you work toward mastery of the basics, stay abreast of novel approaches that offer a treasure trove of techniques that will assist you in confronting change.
In the end, change can make your project better. Often the best ideas emerge in the late stages of the graphic design phase, originating as an unwelcome "suggestion" from a client that threatens your stranglehold on the scope. As you prepare to embrace change, be forewarned that its inevitability is not an excuse for vague or shabby specifications. A clear and well-defined set of requirements will go a long way toward managing the client's expectations. Solid specifications will provide a baseline against which you can measure the costs of future enhancements and a valuable addition to your company's knowledge base. If you've done a thorough job of defining the product, you will enjoy a smooth transition into the next phase—creating the plan.
EXTREME PROGRAMMING
Alex Cone, CEO of CodeFab Enterprise Development (http://www.codefab.com), talks about using Extreme Programming to move beyond the traditional methods for managing scope.
This interview was conducted in October 2001 over a few margaritas at Tortilla Flats restaurant in New York City's West Village. The restaurant served up a spicy mix of retro 1950s paraphernalia and Mexican home cooking to local patrons. The booths were packed with art gallery owners, bikers staggering in from meat packing district bars, and Silicon Alley computer geeks taking a break from their labors in the raw industrial loft spaces that litter the neighborhood.
After a distinguished career developing back-end applications for Wall Street trading systems, Alex teamed up with some of the top developers in town and set up shop in a warehouse space in 1997. Since then, his team has abandoned traditional software development methodologies to produce innovative applications for clients like Apple Computer and Standard & Poor's.
So we're writing a book about how to get Web projects done. Any advice?
When we started CodeFab, the first thing we did was look back at most of the software development projects that we'd done over the last 10, 15 years, and conclude that most of them had failed.
You mean failed from a process point of view?
They failed from the point of view that they didn't get finished in the timeframe or within the budget, and we generally weren't happy with the end results. We wanted to know why. We wanted to actually finish projects and do them the right way. By and large we've been successful over the last four and a half years that we've had this company. At the two-year point, I had already finished more software projects than I had in the previous 15 years. I actually finished them and delivered them to a satisfied client: They were a success.
In most of my experiences on Wall Street, you started some grandiose project, people worked on it for a while, and then all the significant players left. It was never finished, things got changed along the way, and you never saw a completed piece of software that matched what you set out to do.
We started examining the current wisdom on software development and why software projects failed. At that time the point person was Steve McConnell, who had written a number of good books, including Code Complete, Rapid Development, and The Software Project Survival Guide. There was some really good thinking in there, but much of it focused around the cost of introducing changes into your specifications and controlling the scope of your project over time, as well as management issues in terms of keeping on track with what you're doing and the importance of having more or less complete requirements up front. This works very well if your focus is to be able to do fixed-scope projects.
The traditional approach was to do a whole bunch of work against a set of specifications and deliver what was in the specs before moving onward. It seemed like a good way to go, and it certainly provided good backup in terms of a contract with the client. When you're two weeks into the project and the client wants to change something significant, you can say, "Look at the cost associated with making a change at this point." However, by and large this didn't really work for us. We had some notable failures, and part of it was our fault and some of it was the fact that this development model really didn't fit with the Web. One of the primary tenets of the standard process was that you can know at the start of your project what it is that you want to do, what you'll need over the next 6 or 12 months, and that those needs will remain consistent over that time period. And this has turned out to be completely unrealistic. It was also founded on the idea that you could induce a client to actually describe in sufficient depth and detail exactly what they wanted you to do up front, before any work had been done yet. And that also turned out to be impossible. You go out of your way to discourage the client to change the project in midstream. You needed a change control committee, you billed them extra fees for changes, and so on. That tended to foster a very bad working relationship with the client.
In fact, this adversarial relationship was the source of our biggest problems. Bascially, it forces you to have a huge fight up front with the client, wherein you wrestle over the features and the specs and the cost and so on and so forth. The client is trying to get the cost down as far as possible, while you're trying to limit the features. You're trying to get detail out of them when they don't want to be detailed, and then you have this huge fight before you even start the project.
Once the project gets going, you have another big crisis every time the client has a brilliant idea about how they would like to do something. This is because the client wants you to accommodate the change at no cost. Naturally, you show the client your documentation, which proves that the new feature is going to cost a lot of time, energy, and money—so you think you should be paid for it. The brawling continues.
Finally, when you finish the project, you have a big fight at the end. "We're done." "No, you're not." "Yes, we are—see, here are the specifications showing that we did exactly what we said we were going to do." The client is only trying to be financially prudent. This is their opportunity to get you to do more work for free by saying, "Finish this, change this slightly, try and do this," but you're trying to be financially prudent by saying, "Look, if we do another development hour, that is costing me money, and you're not paying me anymore for doing that, so we have to draw the line." So after you wrestle each other to a total standstill, you're probably not in a good head for doing the next project with this client, and he's not psyched to work with you, either.
So the main problem you've identified with the standard methodology is that the customer does not really know what they want, and, to make matters worse, you're working with a new technology that is changing rapidly along with the business environment.
Right, and you're starting off with some assumptions that are fundamentally unreasonable. You can't know exactly where you want to land. This isn't some kind of a ballistic missile. This is like trying to drive to Boston by pointing the front wheels of your car exactly in the direction of your destination address in Cambridge. Then you take your hands off the wheel and just press on the accelerator and hope that you'll get there. That isn't how you really drive to Cambridge. Making such complicated journeys is about midcourse corrections and about being adaptable. The traditional process fundamentally rejects change rather than embracing change.
The only advantage of the standard methodology is that it fits well within the corporate consulting mindset. The corporate client says, "We want to have this e-commerce Web site up by June. I've got to go back to the budget people and get a check for this piece of development. How much will it cost me?" The only people who really succeeded at this were the overpriced consulting agencies who would say, "Okay, then, I'll pull a number out of my butt and completely pad it with a $2 million markup." For a while, clients actually said okay to this, but now we're back to square one.
So if the situation is unworkable unless you pad your estimates by some ridiculous amount, then what do we do?
We really needed to start this whole process off on a completely different foot. One of my developers came to me with this new methodology, started by this guy Kent Beck, who was one of the original developers of Perl. It's called extreme programming, or XP. We started working with this, and we basically came up with a CodeFab version of XP. We worked on a couple of projects with this and found that it addresses pretty much all of our concerns.
The basic idea is—to continue my driving analogy—you agree that we're heading toward Boston, but all we work out initially is how to get to I-95. We'll take the next step once we're on I-95. The client comes in, and you put together a "story" rather than trying to do functional specifications. By trying to write these highly detailed specs, you are struggling to do technological implementation during the wrong phase. You're focusing on the details instead of the goal.
My emphasis on stories is that we want a good customer experience. For example, one story says, "We want to build the application so that the next time the visitor comes to the site we should know what they wanted the last time they came to the site." As you start programming, you come back to this one-paragraph story that describes the idea behind the user experience. You're not focusing on the features. You don't care about specifying the details like "there should be buttons on the left side instead of the right side, and the user should be able to turn on one-click buying with a checkbox."
So the specifications are experience-driven rather than widget-driven. The specifications are just stories about what the user gets out of the experience.
True, but there are a couple of restrictions on the client. The client has to be willing to put somebody on site. The client has to be part of the development team because you are doing short iterations. I can never get more than two weeks away from a working version, but the client has to be able to fine-tune things all the time. For example, there may be two different solutions to a problem, so we have to ask the customer, "Which one do you like?" Or we say, "We're at this point, and we could do any one of these three stories next—which one is the most important?" It really requires close interaction on the client, and that person has to be somebody with authority to make a decision. It can't just be somebody who makes a phone call, because we can make a phone call ourselves. This system requires some serious participation and accountability on the part of the client.
The actual process involves a series of prototypes, developed rapidly with immediate feedback from the client.
You're doing just enough to get the story to work. And then you move on to the next story and get that to work. You build this stuff up. You don't think to yourself, "Okay, we need a grand shopping cart infrastructure before we start anything else. We need a network communications layer before we start anything else." You just do a little bit and then get back to work and do a little bit more. Everything is based on making the stories come true. You iterate in the direction that you want to go. When the client comes in on Monday and says, "I had a great idea over the weekend," you can say, "Okay, great, let's do that now."
How do you get scalability into your software design if you are doing a series of one-offs?
They are not one-offs. You are just adding more features to the baseline. They are not prototypes. They are the real thing. You just make the specs into a story, like "We should be able to handle a thousand users shopping at the same time." The trick is to make it work, make it work right, make it work fast.
The major tenet of the whole XP thing is this refactoring. As you get a new requirement, you are not afraid to rewrite the code to accomplish a new goal. It steers you away from a tendency to design some vast infrastructure that you may not need. You avoid maintaining all of this useless code just because you might need it. You use just enough. You refactor and optimize the stuff that isn't good enough. You don't work on any part that isn't actually a problem.
This is a classic problem with all software development. People tend to optimize too early and optimize the wrong thing. If you find out that one spot is 80 percent of your problem, then you put your efforts there. Don't try and solve a problem if it isn't a problem yet. The client is right there, saying "It's fast enough to go live with now." If it's not fast enough to go live with, let's change this one piece. Let's stop adding new features and make this one thing faster.
If the product evolves on the fly, how do you handle billing?
Basically the client hires a team on a burn rate—say, for $150,000 you get these four guys and their project manager for a month. And we just continue iterating, executing stories, testing, rolling out a version for as long as the client wants. When are we done testing? When the client says we're done testing. When have we got enough features built into the product? When the client says we have enough features.
So the client has to manage their own scope!
Absolutely. But they get lots of short-term feedback. They can take shortcuts if they want to. They can say, "We need to go live with this feature, so we'll just iterate in that direction until we get something that is good enough." They can say, "Now I want to work on the user interface. Now I want to work on the communication with the CRM software." The client can really drive the process.
Since the development team has been doing a series of short-term iterations, they are never more than two weeks away from a working version. The client always has something to play with. The client can always refer to a working version, which allows them to base their ideas and suggestions on a realistic model. Plus, we do other things. We integrate unit tests into everything right up front. You cope with the story, write your first test and run it, and it fails because you haven't written any software. You write the software until it passes the test. If you want to make some changes, you re-run all the tests. If the code passes the tests, then you have a piece of software that does all the stories you've defined so far. You get much higher quality software that way.
We also do a lot pair programming, which is very nice. One person types, and the other person navigates, and we keep rotating people around. This way everyone sort of takes ownership of all of the code. Everyone has a piece of developing the product. It's no longer "This is the JavaScript guy; this is the guy who does all of the database stuff." Everybody gets a piece of everything, and you don't have to study a bunch of documentation to figure out what the other guy did because you worked on developing it. You have the developers volunteer to take on certain tasks. You write them up on 3 ´ 5 cards like you saw on the wall, and people volunteer to do them. This way the client isn't paying for twice as many programmers as they need, and generally after a month or two they are completely sold on the process.
The client feels like part of a team is their team and that they've come up with an idea and have seen it turned into reality very quickly, as opposed to fighting through some change control committee and then six months later finding out whether the feature was useful or not. My experience is that clients change what they wanted, or even what they thought they wanted, all the time. The changes don't necessarily have anything to do with what they were originally thinking.
How do these clients manage the overall budget?
They basically have to accept a degree of uncertainty. It usually takes them a month or two or three to realize that we've got such velocity on this project that it's worth the ambiguity.
In the traditional model, the guy who is writing the check never looked over a set of detailed requirements anyway. The budget guys are just trusting the client manager to pick the right features and do the right implementation, so we don't need to be that much more specific. Either they trust him or they don't. And if there are specific requirements that can be conveyed to us up front, so much the better.
Nonetheless, that person continues to be responsible all the way. And it eliminates the classic "It's the consultant's fault," which generally was just a smokescreen for a person who could never make up his mind about what he wanted.
I guess the client sells this model to their manager by virtue of its flexibility.
Right. Basically the client adds features until they are happy with the feature set or until they can't add more features within the budget. They're deciding what the next feature will be and what features do not make the cut because they will take too long to develop.
We want to shift the burden of expectations and prioritization back to the client and make someone a proactive participant in the process rather than having a project manager who is placed in an adversarial relationship with the client. The client gets a lot more responsibility out of this process, and they also feel tremendously empowered.
How does the project manager's role shift in this relationship? They are not necessarily managing deliverables because deliverables are being established by the client in concert with the developer.
The project manager sort of glues everything together. They are responsible for getting the stories and the tasks written up and estimated, coordinating who is doing what where, and also overseeing interaction with the QA people. Since you start with unit testing right away, QA people get very involved. And there is still the matter of putting together project documentation that has to go through the client's approval process, so the project manager does all that. Plus the project manager is still the primary point of contact for the client. They make sure we have the right documentation and document status reports on what was actually accomplished. Yes, there is a board with a bunch of 3 ´ 5 cards on it, but there needs to be more formal status reports conveyed.
I love this process because I know I am getting paid fair value for the developer. They work as long as the client wants them to work, and they are being paid for it. If the client wants more developers or less developers, we can adjust. I avoid the typical scenario wherein we're working like mad for the last three months, for the final one-sixth of the overall project payment. And what's more, you are in such a perfect state to go on to phase two.
Also, you're in a perpetual brainstorming mode, a creative process as opposed to being on an assembly line process. Developers often feel like they are on the receiving end of a directive, and if you're on the assembly line, you've got to churn out whatever the directive tells them to churn out. There is no creativity.
With this system the developers are empowered to volunteer to take on this or that story and estimate tasks. Then you reiterate, compare the estimates to the actual time, and calculate your velocity or your fudge factor and estimate accordingly. You get more and more accurate about estimating over the course of the project.
Alex, what's your definition of a story, and how do you measure it?
You're trying not to describe technological implementation so much as what the user does and what outputs he receives. No longer than a paragraph. Something that can be easily communicated so that we can say, "Can we make this come true? Do these features make this true?" There is a strong emphasis on using metaphors. You want to be able to give people a good sound bite so they really get a feeling for what this is supposed to do.
Give me an example of a story from one of the cards that would appear up on the wall.
We're doing this distributed publishing product for these Japanese guys. It's basically a Japanese car magazine, but they have thousands of distributors and editors who are on the road all over the place. They want to put a contributor on the road with his laptop. He should be able to upload an article, upload the images that go with the article, and do it all remotely from a laptop over a wireless connection. The users should be able to search for an article by title in English or in Japanese, and the user should be able to find articles pertaining to cars by the model year. That's not saying anything about what their search algorithm is or what the tool is that allows them to upload pictures pertaining to the article.
The stories can change over time. We started out with a Web-based solution. Then we came up with a solution that integrates directly into Microsoft Word, but the story continues to be true. The developer might break the story down into tasks and provide a broad effort estimate for the story. And, of course, everything continues to be recorded on 3 ´ 5 cards, and you work on things until they are done. Then you mark them down and put them up on the board. The project manager is measuring the velocity by saying, "You did that in half a week instead of two weeks."
What makes a good project manager really great?
Good people skills, good organizational skills, willingness to be perceptive about the client and the client's mindset, good ability to digest fairly complex technological issues and translate them into something that the client can really inhale, good ability to work with the employees and balance all the egos. There is still a lot of developer ego management. Making sure that two people don't pair with each other all the time or that nobody leaves anybody out in the cold, making sure that developers are not spinning their wheels for too long on a problem and that they are respecting decisions that were laid down about how we are going to do this or that or the other thing. You are still doing client management. You are still doing status reports. All the clients are different, and they all want weird stuff.
One of our top project managers is a nationally ranked pinball champion and often competes in championships. One of my developers brought us XP and insisted that we use it and is fanatical about it. He keeps sending e-mail messages with sound bites about why this is good or bad or evil or whatever else. But I am actually trying to encourage developers to be in this mode. Part of this is like trying to be a housing coordinator at a college, trying to match roommates.