J2EE Software Development Methodologies

This chapter is from the book 

It has been stated many times that developing software is an art, and in some ways it is—you can look at it from varying angles and perspectives, and you will see a hologram of methodologies that can be used to implement a solution. However, if you sift through the complexities presented by the hologram, you will see that there is a core suite of activities or phases that are common to many of the methodologies, which can be applied toward developing J2EE solutions. These core phases are illustrated in Figure 2.4.

Figure 2.4 The core phases of a J2EE software development effort are common to most of the methodologies that can be applied toward J2EE development.

They are

• The Project Initiation or Feasibility Phase

  Is a pre-project, due diligence exercise to verify the validity of a J2EE software development effort. This is accomplished through the discovery of high level requirements, technical and operational complexities, risks, costs, and lastly a proposed project plan. The Project Initiation phase defines the value proposition for the J2EE solution.

• The Requirements Phase

  Is an iterative exercise that captures the needs of the stakeholders and end users, as well as capturing a clear definition of the problem domain and scope.

• The Analysis Phase

  Is an iterative exercise with the Requirements phase, where business analysts develop an understanding of the problem domain and scope, and then begin to conceptually develop a solution.

• The Design Phase

  Is an interactive exercise with the Analysis phase, where architects and developers begin to develop a technical and conceptual solution based on the results of the Analysis phase.

• The Development or Construction Phase

  Is an iterative exercise with the Design phase, where developers begin to program and develop J2EE objects and services, using the software design proposed in the Design phase.

• The Testing Phase

  Is an iterative exercise with the Development phase, where each logical unit of code (EJB, Servlet, JSP, or Bean) is tested. Upon successful testing, it is deployed for integration testing with other aspects of the system, which have also completed unit testing.

• The Implementation or Production Phase

  The final phase of the software development effort is to deploy the J2EE solution into a production-ready BEA WebLogic Server. A final production test is required to ensure that all J2EE objects and services are working as expected and interactions with any external (legacy) systems and databases are operational.

The whole purpose of following a software development methodology is so you have a methodical set of process or tasks that, if followed correctly, will deliver the system solution you and your end users coherently envisioned. By understanding these phases in the context of a methodology, and the activities they entail, you will be well informed to validate whether your organization can successfully embrace it to develop your J2EE solutions.

Selecting a Software Development Methodology

There are plenty of software development methodologies available; each one being unique in the way it performs the core phases of a SDLC. The methodology that you decide to adopt should be well-documented and supported in order to act as a guide when in you are in doubt. It is important to review the origin of the methodology, since it can be developed in-house by large consulting companies, fostered by consultants/authors, or crafted by software product companies. Based on the origin, you will quickly understand all the support infrastructures available for that methodology. You will also be able to evaluate whether there is a cost factor involved for using a specific methodology or obtaining any support.

Another important factor for selecting a methodology is whether it supports the type of system you intend to build. Since this book is focused on the BEA WebLogic Server, there is a high probability that you will be developing a Web-centric system. As a result, not only will you be dealing with a project staff quite different from the kind of project staff involved in most software engineering endeavors (graphic designers and HTML developers, and so on), you will also need to include a usability study throughout the SDLC.

At a high level there are two types of methodologies—Predictive (Heavyweight) and Agile (Lightweight).

Predictive methodologies, also known as heavyweights due to the sheer overhead that comes with using them, such as the Rational Unified Process (RUP), are full lifecycle methodologies which posses the following attributes:

• They are tools-based, which means that a modeling tool is required to support the adoption of the methodology. As in the case of RUP, you can use either TogetherSoft's Control Center 5.5 product or Rational Rose 2001.

• The methodologies are extremely well documented for the phases of a software development lifecycle.

• They are either process- or architecture-driven.

• The methodologies are template-driven, which means that document templates guide the use of the methodology. All you have to do is execute according to the guidelines, and fill in the blanks of the template. Hence, the project will produce a large set of documentation, all of which will need to be read, signed off, and managed.

• They encourage knowledge collaboration within the project team, using a common vocabulary.

• They are flexible and modular in nature, suiting small- to large-scale projects.

• They are developed and supported by software houses.

• The software development methodology will need to be purchased at a cost.

Agile methodologies, such as BEA Systems Accelerated Process and SteelThread, eXtreme Programming, SCRUM, Feature Driven Development (FDD), and Dynamic System Development Method (DSDM), possess the following attributes:

• They are not bound to one specific modeling tool.

• The project staff is not required to fill out specific templates.

• Depending on the methodology, the software development guidelines can be documented anywhere from a very general to a detailed manner.

• The methodologies promote Rapid Application Development (RAD) philosophies, focusing on a quick time-to-market approach for the solution.

• They are supported by formal consortiums or public communities of practice, which provide revisions to the methodology for public appraisal.

• The Agile software development methodologies are either typically free to use, or the supporting documentation may be purchased for a small nominal fee, as in the case of DSDM.

The following sections will outline the more popular methodologies that you may want to initially consider in your selection process. Since a detailed analysis of each methodology would be beyond the scope of this book, it is recommended that you first develop an overview of the methodologies discussed and if need be, investigate them further through the resource links provided at the end of each methodology section.

Since this book is focused on BEA Technologies, the BEA Systems methodologies will be discussed in some detail.

BEA Systems Accelerated Process (Project Initiation Phase)

The Project Initiation or Feasibility phase of a project validates the value proposition of developing a technical solution for a problem domain, and creates the environment for the project to be successful. The bottom line for this phase is to decide on whether the project is a "Go" or "No Go." If it is a Go, then you must consider what are the initial project plan, resource requirements, software development methodology, and the technology that will be applied toward the solution.

There are two main problems in developing software today. First, sometimes this phase is completely omitted, causing the software development lifecycle to begin with the Requirements phase. The problem with this situation is that there is no substantial due diligence that the project delivers value or will be successful. Second, this phase often takes too long and the project goes over budget, suffers from poor morale, and then dies a slow death.

In order to validate the viability of a project and arrive at a point that decisions can be made, there will be some questions that need to be answered. Obviously, each organization will have their own set of questions before a project is initiated, but here is a sample of questions that should be answered:

• What is the problem domain, in a clear uninterruptible format—for example, does it hold a value proposition for a solution?

• What are the needs from the business or problem domain—for example, a vision or mission statement?

• What are the constraints the project will have to operate within—for example the political, technical, and cultural environment?

• What are the corporate or organizational rules that will apply to the project—for example policies on spending and technology?

• What is the longevity of the solution, and what solution (operational or technical) will scale to the end?

• At a high level, what are the technical and operational requirements for the project—for example scalability, availability, and scheduled usage time (24*7)?

In order to move forward with a stable footing on a project, it is extremely important how you execute this phase. Remember, you and your organization must enter a project with a high confidence level that it will be delivered successfully with a minimum number of surprises.

Most OO methodologies will concur that this phase is required; however they fail to explain how it should be conducted, and hence how the deliverables can be achieved. BEA Systems, cognizant of this void in systems development efforts, and in an effort to ensure projects utilize J2EE and their WebLogic application servers, have developed an approach called the "Accelerated Process" (AP), which is provided through their Professional Services Group.

The objective of the Accelerated Process is to execute the project initiation phase in the shortest and most feasible time frame possible, without sacrificing the quality of any of the following deliverables:

• A Feature Set Document (FSD), where system requirements are defined into categories in terms of their respective release schedules.

• A Project Strategy, which describes how the project should be conducted and what software development methodology best suits the project.

• A Project Plan to develop the solution, which includes the project phases, schedules, resources requirements, and other associated costs.

The Accelerated Process is principally comprised of the following activities:

• API Event—Accelerated Project Initiation

• ARM Session—Accelerated Requirements Method

• ATFA Session—Accelerated Technical Feasibility Assessment

• CVM Event—Customer Validation Meeting

• ARRP Session—Accelerated Risk Reduction Planning

• RVM Event—Risk Validation Meeting

• APP Session—Accelerated Project Planning

The sessions are highly structured and participatory activities. The events are informal meetings, all of which are conducted by an experienced facilitator, whose responsibility is to guide the sessions and events to ensure the outcome is delivered successfully (see Figure 2.5).

Figure 2.5 The BEA Systems Accelerated Process is composed of highly structured and participatory events.

The Participators of the Accelerated Process

The participators of this process fall into two categories—Customers and Suppliers. Customers should be empowered people or Subject Matter Experts (SMEs) from cross-sections of your organization that will derive a direct benefit from the J2EE and BEA WebLogic solution.

Alternatively, Suppliers are technical personnel or experts whose responsibility is to provide input into the technical decisions that need to be made during this phase, and potentially for all subsequent phases of the SDLC (analysis, design, development, test, documentation, and training). Examples of Suppliers include architects, lead developers, database architects, the project manager, and any external personnel representing the technical vendors for the project, for example BEA Systems, as in the case for BEA WebLogic Server.

Accelerated Project Initiation

The API is the first phase of the Accelerated Process. This is a very short event; typically in the context of a meeting, the executive sponsors, stakeholders, and project managers define the project in terms of its scope and vision. The result of this meeting leads to the development of the framework and a decision where emphasis will be placed in the subsequent sessions and events that comprise the Accelerated Process.

The outputs of the API event include

• An identification of the project sponsors and other authoritative decision makers.

• Establishing a vision, the objectives, and scope of the project.

• Establishing the business and functional success metrics for the project.

• An initial schedule of the required Accelerated Process events and sessions.

• A preliminary list of project Customers and Suppliers with names and defined roles.

• Identification of any related Customer documentation pertinent to the project.

As in all the events and sessions in the Accelerated Process, the outputs lead into the next phase.

Accelerated Requirements Method

The ARM is a very formal facilitator-based and documented session with the Customers of the project. It typically lasts anywhere from a few hours to a maximum of a few days, depending on the complexity of the project. The objective is to gain consensus and alignment at a high level on the business requirements for the project, without any emphasis on the technological feasibility of the solution. The goal here is to concisely, but accurately, state the problem that needs to be solved—the what is and the whys, not the how is. Even though requirements for the system are gathered at a high level from the Customers, this exercise is not a replacement for the Requirements phase. In this phase, the requirements will be gathered in a more formal and due diligent manner across the spectrum of the project's scope.

The output of this session is a real-time list of functional requirements, as proposed by the customers, with the following value-added descriptors:

• Categorization—Categories are developed by grouping requirements according to their likeness.

• Benefit Points—Define how a functional requirement will profit the target business or problem domain.

• Proof Points—Represent one or more positive statements that prove a functional requirement has been met.

• Annotated Commentary—Is a set of assumptions, issues, action items, and comments made about each functional requirement.

• Prioritization—Prioritization artifacts identify the precedence or importance of each functional requirement defined in conjunction with the other functional requirements.

The outputs of the ARM session become the inputs to the next AP event, the ATFA.

Accelerated Technical Feasibility Assessment

The ATFA is very similar to the ARM, except in ATFA the Suppliers are in the spotlight. Again, this is a very formal facilitator-based session with inputs directly from the ARM session. Ideally, the Accelerated Process facilitator will coordinate this session with a technology expert/architect or evangelist from BEA Systems or the J2EE technology world.

The objectives of this session are to solidify the scope of the project through discussing the technical feasibility concerns, technical requirements, overall project approaches, and key technical assumptions which stem from the business requirements that were proposed by Customers in the ARM session. Depending on the complexity of the project, the duration of this session could be anywhere from a couple hours to a few days.

The delivered outputs of the ATFA are consolidated with the API and ARM outputs to form a document formally known as the Combined Findings Document (CFD).

The outputs of the ATFA include the following:

• An early high-level technical architecture diagram, providing the Suppliers with an initial vision of how the solution will be designed.

• A list of assumptions made by the Supplier team regarding the project.

• A list of issues or concerns identified by the Supplier team which could potentially impact the project.

• A list of potential functional requirements made by the Supplier team that were not collected during the ARM session.

• A list of non-functional requirements the Supplier team requires in order to begin to develop the potential solution.

• A technical assessment for each of the functional requirements identified in the ARM session, which ensures synchronicity between the two groups, the Customers and Suppliers.

The output of this session provides input into the next AP event, the Customer Validation Meeting (CVM).

Customer Validation Meeting

During this meeting, the facilitator of Accelerated Process and the advocates from the Customer team review the Combined Findings Document. The output from the ATFA is included in the Combined Findings Document.

The objective of the CVM is to identify any technicalities from the ATFA session that may affect the scope or complexity of the project. Since the Accelerated Process is really gauged to be customer-driven, it is the Customers who decide whether to accept or refute the conditions and requests made by the Suppliers in the ATFA.

Accelerated Risk Reduction Planning

The ARRP is a highly formal session with the Supplier team to identify, assess, and document the risks of the project in the Combined Findings Document. The starting point for this session is a review of the Combined Findings Document and the results of the Customer Validation Meeting. By focusing on risks involved prior to any planning exercises, the risks can either be mitigated or contained through inclusion strategies, ensuring the initial project plan has a safe start. The outputs of the ARRP session directly affect the Project Plan.

The output from this session is a documented list of risks in the Combined Findings Document, their associated consequences if not managed, any plans of mitigation, and any proposed containment strategies. These outputs then feed directly into the next AP event, the Risk Validation Meeting (RVM).

Risk Validation Meeting

In this event, the advocates from the Customer team review and assess the key risks in the Combined Findings Document, as proposed in the ARRP session. The advocates from the Customer team can accept the risks or request further clarification from the Supplier team, but they cannot refute any risks.

The output of this meeting, which typically lasts a day, is documented in the Combined Findings Document. It is used as a basis to develop the initial project plan.

Accelerated Project Planning

In this event, the Accelerated Process facilitator in conjunction with the project manager begin to review the Combined Findings Document, which includes output from all previous AP sessions.

Their objective is to identify the project requirements with their associated technical metrics for realization, and begin to formularize the following:

• Project Structure—Project tasks and their estimated start and end dates, and any milestones.

• The Software Development Process—A methodology, which should embrace the following activities:

• Implementation of best practices or standards for consistent development

• Architecture modeling

• Object and code reuse

• Code inspections

• Software change management

• Production support

• The project management approach.

• Augmentation to the project using consultants.

• Any training that will need to be conducted to educate the project staff.

By determining the estimated project tasks, and associated start and end dates, an overall project duration can be developed.

The outputs from this event are as follows:

• A Feature Set Document (FSD), where system requirements are defined into categories in terms of their respective release schedules.

• A Project Strategy, which describes how the project should be conducted and what software development methodology would be best suited to deliver a rapid time-to-market solution.

• A Project Plan to develop the solution, which includes the project phases, schedules, resource requirements, and other associated costs.

The two factors that will greatly influence the success of the APP event fall on how well the project plans and strategies have adapted themselves for component-based development. It is imperative that the touchpoints described in the following sections exist quite prominently in the project plan and strategy.

Refactoring the Project Plan into Binary Deliverables The project plan should be refactored into smaller incremental and iterative phases with milestones or binary deliverables.

A binary deliverable is an executable deliverable that has one of two states—done or not done. Since Analysis and Design can never be shown to be complete, they should not be considered binary deliverables. Ideally, a binary deliverable should be some software code which demonstrates a requirement the end-user (Customer) can relate to. For a J2EE system, this will include the presentation, server-side, and even data tiers of the system.

Every organization is different, so it is up to the project manager to define the duration between milestones that is acceptable to the end-users, without giving the impression too much time has passed without anything to show for it.

Iterative and Incremental Software Development Practices These two philosophies will stand the test of time in the development of component-based systems. First, you must acknowledge that you will not always get things right the first time around in developing software. Therefore, you will have to return to certain activities, but knowing more than you did initially enables you to be closer to getting it right. This is the concept of iterative development, which recognizes a single return to an activity is unlikely to result in complete success. As a result, each activity is repeated many times to refine the deliverables.

A software development process should never attempt to build an entire system in one monolithic effort. It should be partitioned into binary deliverables, each with its own independent, parallel streamlined effort—individual sub-project plan. The process involves each binary deliverable being developed, unit tested independently, and then integrated into the full system. This concept is known as incremental development.

Iterative and incremental software development practices should not blind the project members to the overall software development objective. This is quite common, as people are so consumed in what they need to deliver that they forget about the rest of the world. Hence, it is important to keep people connected in how the project is progressing as a group, not only so they understand there are other aspects of the project, but also to promote knowledge transfer from anything learned, positive or negative.

Continuous Project Control Through Feedback In order to control a software development effort, you need a natural feedback communication mechanism. The term natural is used because if it is not natural, there will be resistance in providing feedback over time.

The project manager is the person solely responsible for managing the project, and will consistently need to measure its progress; identify new risks and provide counter measures; compare that progress against the plan; and then fine-tune the development parameters to correct any deviations from the plan.

There is no given rule to what the feedback communication must be; for example, meetings, presentations, or artifacts such as status reports are normal means to provide feedback. The bottom line is that it must be acceptable to the people that will provide it; otherwise it will not work.

Realistic Deadlines on Milestones Unrealistic deadlines can cause unnecessary stress to everyone involved in the project, especially the development staff that has to produce a tangible product to show the user at the end of each milestone. This inevitably renders poor quality in the work and morale of the software development effort.

To prevent such environments, it is important to assess the iterative cycles of development, and reflect the true measures of effort for the delivery of each milestone into the project plan. In addition, during times of stress, it is important to keep people focused and involved in their domains of expertise, thus gaining the maximum return on their time. For example, developers should not be gathering requirements in times of stress; that should be the role of the business analysts.

The outputs of the APP become the inputs to the next AP event, the Project Commitment Meeting (PCM).

Project Commitment Meeting

In this final meeting in the Accelerated Process, the Supplier team, including the project manager, formally presents their plans of executing the project and developing the technical solution to the Customer team, using the output from the APP.

During this meeting, the Customers will provide challenges by asking qualifying questions of the Supplier team, in order to make a decision on whether the project will be given the "Green Light" to proceed. Once agreed, the Project Plan, Feature Set Document, and the Project Strategy are all leveraged into the actual software development effort, thus providing a high confidence level for success to the overall project.

BEA Systems SteelThread (Architectural Prototyping)

One of the key architectural risks you will encounter in today's J2EE system development efforts is the question of integration with other systems.

If your integration is going to occur inside WebLogic Server, it is going to be Java-based, and more than likely it will be successful, given that it is on the same platform and all you will need to do is tap into the appropriate interface with the right information and do some through testing. Tuning BEA WebLogic Server will take care of the performance and scalability issues.

However, if you are developing a distributed architecture which will include legacy, database, and vendor-based solutions outside the realm of WebLogic Server, the immediate question is the feasibility, scalability, and performance of the distributed architecture. It is imperative you validate all distributed architectures up front before any concentrated development efforts occur.

A BEA Systems SteelThread, rendered as a service through their Professional Services Group, is a prototyping methodology that embraces the concept of quickly developing an end-to-end "thread" of technical and procedural functionality the distributed architecture will need to support, as illustrated in Figure 2.6.

Figure 2.6 SteelThread promotes the idea of developing a single end-to-end "thread" or functionality the distributed architecture will need to support.

Ideally a SteelThread should be a single requirement that spans the complexities of your distributed architecture. Metaphorically, a requirement should be an inch wide and be proofed a mile deep, as illustrated in Figure 2.7.

Figure 2.7 In the SteelThread prototyping approach the thread should be a single requirement that spans your entire distributed architecture.

For example, a thread may include the presentation layer (JSP, HTML), server-side Java (Servlets, Beans, EJBs), and span all the way to the legacy and data layers.

An important aspect of the SteelThread prototype is that it is built with a mindset that it will serve first as a system design framework, and second as a foundation for future development. A SteelThread is not a throw-away, as the prototype term can sometimes imply. If the SteelThread is not successful in its objectives, it will at least serve as an excellent early proofing method for the design of a system. At the same time, another approach for the distributed architecture will need to be devised and executed using the SteelThread method.

The benefits of using a SteelThread are

• You are able to realize early the capabilities, risks, and constraints of the distributed architecture. This will in turn enhance your understanding of the total system as you move closer to the actual development phases.

• There is an extensive amount of knowledge transfer from BEA Professional Services to your development staff around the BEA WebLogic Server 7.0 capabilities and its facilitation of J2EE distributed architectures.

• After participating in a SteelThread effort, your development staff will have experienced an end-to-end solution of a requirement. Hence, they will be better poised to develop the J2EE solution in parallel efforts.

SteelThread Prerequisites

Surprisingly, the Analysis and Design phases do not need to be completed for any SteelThread activity to begin. However, since the objective is to derive an end-to-end solution for a specific requirement, the duration a SteelThread will depend on the quality of the following factors, as illustrated in Figure 2.8.

Figure 2.8 Several factors influence a SteelThread effort.

• Defined technical requirements.

• An understanding of the business requirements by the development staff.

• Project knowledge of the interfacing systems (Integration).

• Availability of vendors and personnel from the interfacing systems (Integration).

• The availability of any environment to design, develop, and test the SteelThread.

The better the information that feeds into a SteelThread, the quicker the pace for its execution. However, the less input you provide into a SteelThread, the more work the SteelThread will incur to gather the base inputs.

eXtreme Programming

eXtreme Programming or "XP" is a lightweight software methodology that was developed by Kent Beck. Since its inception approximately five years ago, XP has evolved and inspired a developer-centric revolution. The XP methodology has its roots in projects where requirements are prone to change; development risks need to be mitigated to ensure success; and there are a small number of developers within an extended development team.

Everyone who participates in XP is considered an integral part of the team, including the advocates from the business. The "Whole Team" concept includes the following roles:

• Tracker—Is in contact with the XP developers regularly, and provides a roadmap action for any concerns or problems.

• Customer—Plays the role of the subject matter expert, and has the responsibility and authority to explain the requirements and set priorities as to which requirements are designed and developed.

• Programmer—Estimates the length of the development and testing cycle, and implements one or more requirements into a functional piece of software.

• Tester—Implements and runs the functional and acceptance test of the software code.

• Coach—Ensures the project remains focused and does not deviate from being "eXtreme."

• Manager—Is the administrative arm of the group, who for example arranges meetings and ensures they are conducted to meet their objectives.

• Doom Sayer—Shouts out when there are severe problems with the project.

The success of XP in today's methodology wars has been how it has evolved using the following core principles:

• Simplicity—The design of the system is kept deliberately simple and clean, thereby delivering the software your customer needs, when it is needed.

• Communication—XP emphasizes teamwork between project managers, customers, and software developers, all being part of the XP team.

• Feedback—XP developers communicate regularly with their customers and fellow programmers. Through testing their software early with continuous feedback, XP developers can deliver the system to the customers as early as possible, and implement changes as suggested.

• XP stresses complete customer satisfaction.

• Courage—XP empowers developers to confidently respond to rapidly changing customer requirements and technology, even late in the life cycle.

Using these principles, XP itself is conducted through a few rules and a large number of software development practices, thus establishing a methodology that is quite streamlined toward developing software. It embraces the rules and practices that promote the development of creativity, speed, and quality, and overlooks anything that can seem too complex to practice or an overhead for the development effort; hence the reason why this approach was coined the name "eXtreme Programming" and is considered extreme by most software development traditionalists. For example, XP eliminates the unnecessary artifacts of most heavyweight processes (formal status report, large volumes of requirements and analysis documents, and even UML diagrams), which can slow down and drain the development staff.

XP, as you will discover in the following sections, is very development- and test-centric. Through developing test scenarios and associated prototypes, the software solution is evolved from concept to actual code to refined design. It is this test-first-and-design-later approach that makes XP so efficient, as illustrated in Figure 2.9.

Figure 2.9 XP is very strongly dependent on iterative testing.

The 12 Core Practices of eXtreme Programming

In order to gain an introduction to XP, you will need to understand the 12 core practices of eXtreme Programming, which can be broadly described through the following:

• The Planning Game Practices, Small Releases Practices, and Customer Test Practices

  The XP team embraces a simple form of planning and tracking that allows the development of a solution in a series of small, fully integrated releases, in close consensus with the customer.

• The Simple Design Practices, Pair Programming Practices, Test-First Development Practices, and Design Improvement Practices

  XP developers work together in pairs, continuously improving the design of the solution through a repetitive testing ritual.

• Continuous Integration Practices, Collective Code Ownership Practices, and Coding Style Practices

  Once the initial skeleton system can be deployed, the XP developers will continuously work as a group to keep it operational and maintained through incremental code integration and a consistent coding style.

• Metaphor Practices and Sustained Pace Practices

  The XP team shares the same perspective of the requirements and solution, developing it at a sustained pace.

The Planning Game

The main idea behind this practice is to make a rough plan quickly, and refine it as things become more clear, since both the customers and the developers will evolve their understanding of the requirements and desired solutions as the project progresses. The planning game also emphasizes visibility of the progress through some tangible releases as soon as possible, not only to show progress, but also to validate the projects existence.

This practice focuses on steering the project toward predicting the answers to two key questions:

• What to initially release, and by what due date?

• What to do next?

These are addressed through two exercises—Initial Release Planning and Iteration Planning.

Initial Release Planning

During this exercise, the customer presents the desired requirements of the system to the XP programmers through user stories, which are small descriptions of the features and functionality of the system written on index cards. After the initial user stories have been collected, the customer sorts the stories by prioritizing them into the following three piles:

1. Critical for system to meet its business goals.

2. Will have a measurable impact on the system's business goals.

3. Will make the user happy, but without explicit justification.

The programmers then sort user stories by risk into the following three piles:

1. We know exactly how to do this.

2. We think we know how to do this.

3. We have no idea what this means or how to do this.

By now, each card will carry a priority and risk factors, allowing the whole team to pinpoint the business requirements that require further clarification. The ones that carry a high priority and risk can then be refined through improved requirements or a prototyping exercise. Features with low priority and high risk are suspended until the whole solution has taken shape, thereby allowing the requirement to be better understood.

Once the requirements have been identified, the programmers estimate the level of difficulty and associated costs involved, and then lay out a plan to prototype the requirements into a tangible initial release that the customer can relate to. Initial release plans are imprecise, since the priorities and their estimates are not truly solid, and until the first prototype is built, the release schedules will not be accurately predicted.

After the initial release of a requirement, the knowledge of the amount of effort required is very visible, hence cultivating more predictable release schedules as the project progresses through subsequent prototyping efforts.

Iteration Planning

Within the Iteration planning exercise, the customer provides the programmers with features that will need to be delivered into the software system within a two-week window frame. Using the knowledge from preceding prototyping efforts, the programmers decompose the requirements into a granular roadmap consisting of tasks, time, and costs. The amount of progress made every two weeks is binary—a user story may or may not be implemented as a software component.

Small Releases

A golden practice in XP is to get something in front of the customers as soon as possible. The only technique to enable you to facilitate this is iteratively developing small releases of the system, each one having a bearing on a user story that the customer has provided. Some XP Web projects release to their customers as often as daily.

Not only can the customer reap the business value, but also this provides a mechanism for early positive and negative feedback to the whole XP process.

Customer Tests (Acceptance Testing)

As part of providing user stories to the programmers, customers must also provide a means to test whether the desired features are functional in a specific manner. This acceptance testing criteria is embossed into a programmer's efforts, causing all releases to be validated prior to customer viewing. This not only saves the customers time, but also illustrates that progress is positive. The customer is encouraged to be available as much as possible to the XP programmers.

Simple Design

In an incremental, iterative process like XP, a simple consistent design philosophy is critical to the formula for success. As each project will be different, the definition of a simple design can be quite nebulous. Probably the best definition can be derived from the touchpoints provided by Kent Beck, where a simple design

• Runs all the programmer's tests

• Contains no duplicate code

• Clearly states the intent of the programmer within the source code

• Contains the fewest possible classes and methods

The idea of a simple design is to provide exactly what the customer has requested—no more, no less. Deviations from simplicity can cause extended release schedules and undue complexities in the actual deployed solutions.

Pair Programming

Of all the aspects of XP, pair programming is probably the most questioned and argued for its productivity. Through the Pair Programming Practice, a user story is designed, developed, tested, and deployed by two programmers sitting side by side, at the same machine. Pair programming involves writing all production code in pairs, sharing a single machine. The notion is that two heads think better than one, while at the same time functioning as a single entity. This ensures that every aspect of an XP release is a collaborative effort between two XP programmers.

One XP programmer typically sits in the driver's seat, designing and coding, while the other (co-driver) sits and watches, questioning any decisions and providing helpful resources as needed. After a release is deployed, the two XP programmers will move either together or separately to assume similar or opposite roles in other XP release efforts.

This arrangement can be quite difficult to fathom, but there are advantages to pair programming:

• Business and technical knowledge transfer is osmotic when two people sit side-by-side.

• An experienced XP programmer functioning as the driver can easily teach the co-driver at the same time as delivering a release.

• Changing paired XP programmers will increase the comprehension of the total system solution.

• The design and source code must be consistently reviewed by one person.

The budgetary personnel in a project who are used to heavyweight methodologies may faint when they learn that two resources are required to sit side-by-side, which can be perceived to be an expensive endeavor. However, industry evaluations have proved that two programmers working together can do a better job than one. An excellent resource to further investigate pair programming can be found at http://www.pairprogramming.com.

Test-First Development (Unit Testing)

XP programmers use the concept of unit testing as a means to design and develop their specific software releases. Unit testing has programmers first write tests, and then create software to fulfill test requirements. Unit testing is a means for testing a thread of a system at any point in time. Even before a line of Java code is written, comprehensive unit test cases are envisioned for each method that will be developed.

Once the unit test cases are specified, development can begin. Source code is written in a test-centric manner, writing enough source code to execute and pass the associated unit test. As a result, software development evolves, preserving the simple design philosophy and providing 100% test coverage on the source code.

Only software that has undergone successful unit testing is checked into a software change management system. The test cases are then automated and collected to a large test set. As software releases are integrated into the larger solution, the test cases are run in an automated manner to ensure nothing is broken.

Design Improvement (Refactoring)

Continuous design improvement, or refactoring as it is termed, is a technique of improving the source code of a unit of software without modifying any of its functionality.

Throughout the XP programming efforts, programmers (the drivers and co-drivers) will be continuously predicting whether there is a better way to implement their software development effort. Programmers will be looking toward providing high cohesion and low coupling in their software, which is typically the trademark of a well-thought-out software solution. Hence, all duplication of any kind (objects, methods, and services) is removed.

Continuous Integration

There is a technical phrase back east, "A software system does not break; it is delivered broken." In the west, this is known as integration hell, where the entire software system is broken and no one knows why.

XP aims to mitigate this horror of a situation by imposing that as soon as the developed system is mature enough to be integrated, it stays integrated, with rebuilds on every new introduction of code and multiple scheduled rebuilds on a daily basis.

There are three primary reasons why XP puts a lot of emphasis on this practice:

1. The XP team collectively becomes knowledgeable on the building process, not relying on inexperience at any time.

2. Problems with newly integrated code can be addressed immediately by the associated developers at hand; the cause of failure for any particular integration effort is obvious.

3. The only code that is frozen is off a working system, which can be rebuilt to provide a stable platform for future integration efforts.

With XP providing a continuous feedback loop to the success of a developed piece of code, it should not end when it is integrated into the large system set.

Collective Code Ownership

There are software development projects today where programming code becomes an emotional asset owned by its author, with all subsequent changes having to go through some approval and coordination process directly with the author of the code. This is such a lengthy and unnecessary process and can cause inter-team conflicts.

XP does not believe in any of this possessive nonsense. Within an XP project, every programmer owns and is responsible for all the code. Therefore, any pair of programmers can improve or modify any line of code at any time, practicing Unit Testing of course. This provides more eyes and minds to review the software as it is developed.

Coding Standard

In order to practice Collective Code Ownership and Design Improvement consistently though an XP project, a coding standard and style need to be in place that all programmers follow religiously. The objective is for all the code in the system to appear as if it were written by one person on the XP project.

Without having any coding standard, it will be extremely difficult to refactor code, switch pairs of programmers to manage other software development efforts, and practice collective code ownership.

The coding standard and style should be discussed and implemented as much as possible before coding begins, to avoid any retrofitting further into the development effort.

Metaphors

Metaphors within the context of an XP project are a glossary of terms that becomes a fundamental part of an XP vocabulary within a project, making it easier for people to converse about how the solution system will work, where to look for functionality, and where to place functionality in the context of the overall system. Metaphors are defined at project launch and prevent ad-hoc names from being created to define aspects of the technical system, such as Java classes, methods and variables.

Sustainable Pace

The XP methodology is a very team-oriented methodology, which implies that the pace of the project is governed by the people who will collectively execute it. There is no point having 70-hour work weeks if only one third of your project team can live up to those terms. It is unreasonable to impose unnecessary hours of work when the work will not be productive or conducive to the success of the project. There have been many studies on the harmful effects of working long hours: stress, poor productivity, coding mistakes, hair-loss, weight-gain and loss, poor vision, metabolic fatigue, and mental burn out are just a few of the possible harmful effects.

XP has a philosophy that the project will be a success if the selected pace of the project can be sustained by everyone throughout the SDLC. If people are having fun on a project, they will naturally work the hours necessary to make it successful. The "whatever it takes" attitude should come from the team, and not be imposed on it.

eXtreme Programming Resources

To get a better and detailed perspective of XP, you should review the following books:

• Extreme Programming Explained: Embrace Change, by Kent Beck

• Extreme Programming Applied: Playing to Win, by Ken Auer and Roy Miller

• Extreme Programming Installed, by Ron E. Jeffries, Chet Hendrickson and Ann Anderson

• Planning Extreme Programming, by Kent Beck and Martin Fowler

• Extreme Programming Examined, by Giancarlo Succi and Michele Marchesi

• Extreme Programming in Practice, by James W. Newkirk and Robert C. Martin

• Agile Modeling: Effective Practices for Extreme Programming and the Unified Process, by Scott W. Ambler

The Rational Unified Process

The Rational Unified Process, or RUP as it is known in the development community, is an excellent example of a predictable or heavyweight software development methodology.

The Rational Unified Process has its roots in the Objectory Process developed by Ivar Jacobson, which was an object-oriented design methodology that focused on development through the use of case modeling. With the efforts of Ivar Jacobson, Grady Booch, Jim Rumbaugh, Philippe Kruchten, Walker Royce, and other people from Rational Software, the Objectory Process began to evolve. The realization of the effort being the Rational Unified Process, which

• Unified the crème de la crème of proven software development practices.

• Embraced UML as the de facto notation for all of its modeling artifacts.

• Addressed the full software development lifecycle from project inception to post-production phases.

The Rational Unified Process is in essence a very comprehensive software engineering process that needs to be followed meticulously, since it asks the following:

1. Who are the members of the project team, and what are their explicit roles and tasks?

2. When do the tasks need to be performed?

3. How do you reach a specific activity or objective?

4. What are the artifacts for each activity (input and output)?

The Rational Unified Process Product

In order to effectively utilize the Rational Unified Process, you not only need to purchase the product from Rational Software, but you also have to purchase a modeling tool that supports the current implementation of UML and the Rational Unified Process. Unfortunately, there is only one to choose from—Rational Software's Rational Rose 2001.

The product itself, as illustrated in Figure 2.10, is Web-enabled, and needs to reside on every project member's desktop while the process is underway. This allows all of the team members to share one knowledge base, one view of the process, and one modeling language reference (UML).

Figure 2.10 The Web-enabled Rational Unified Process interface needs to reside on every project member's desktop.

In conjunction with Rational Rose, you have access to a wide range of features and support resources, for example

• A powerful graphical navigation system and search engine.

• An interactive knowledge base.

• Guidelines, templates, and tool mentors.

• Examples and templates for how to construct UML models in Rational Rose 2001.

• Access to Rational's Resource Center, where you can review white papers, updates, hints, and add-on products, such as the BEA WebLogic Plug-in.

Both the Rational Unified Process and Rational Rose 2001 are available for download for a fifteen-day trial at the Rational Software Web site (http://www.rational.com).

The Principles of the Rational Unified Process

The principles of the Rational Unified Process are all based on the following software development practices:

1. Interactive Development: Through the feedback achieved in every design and development cycle, an evolving understanding of the problem is refined, illuminating potential risks before they can arise.

2. Use Case Driven: Use cases provide the primary requirements of the problem domain, which are then further refined to detailed use cases. They provide a visual perspective of the system by answering who its users are and how they interact with it.

3. Architecture-Centric design: An architectural blueprint of the desired system is built early, and is central to the future development of the solution system. Having a stable upfront architecture paves the road for parallel development efforts, and clearly identifies where and how the software design will be implemented.

4. Risk Management: Encourages the high priority risks to be mitigated sooner rather than later.

5. Software Quality Control: Using appropriate metrics, quality control is built into every aspect of the Rational Unified Process.

6. Software Change Management: The Rational Unified Process is a collaborative effort from a mid-to-large number of people within a team. A high frequency of documents, artifacts, and source code are generated, all of which need to be stored centrally and managed to control any changes.

The Organization of the Unified Process

The Rational Unified Process has two dimensions—Phases and Process Workflows, as illustrated in Figure 2.11.

Figure 2.11 The Software Development Lifecycle as defined by the Rational Unified Process has two dimensions.

The Phases of a Rational Unified Process

There are four distinct phases in a Rational Unified Process SDLC—Inception, Elaboration, Construction, and Transition.

Each phase is composed of a number of iterations. Each iteration constitutes a complete development lifecycle, from requirements to deployment of an executable piece of code, as illustrated in Figure 2.12.

Figure 2.12  Iterative development constitutes each phase.

The first pass through the four phases is called the Initial Development Cycle. Unless the life of the product stops, an existing product will evolve into its next generation by repeating the same sequence of inception, elaboration, construction, and transition phases, which is known as the Evolution Cycles.

A phase is analogous to a milestone, with instructions on how to achieve a given set of objectives, what artifacts to produce, and how to evaluate the quality of the deliverables. Based on this information, a management decision can be made on whether to proceed to the next phase.

Inception

The objective of this phase is to specify a business case and project scope, including initial requirements, costs, and risks for the desired system. If the project is feasible, a plan is developed for the next phase of the RUP, the Elaboration Phase.

To gain an understating of the problem domain and scope, multiple high-level use cases are developed. A use case defines and describes a way an end user (actor) performs a series of steps to obtain a result. Use case diagrams model all interactions between an actor and a system in a single high-level diagram. This allows everyone to be in sync with the intent and scope of domain.

The documents and models that are produced from this phase contribute toward the Requirement Set artifacts, and can include

• A vision of the desired system.

• Use case models of the primary functionality of the system.

• A tentative architecture.

• A project plan for the Elaboration Phase.

Elaboration

The primary objective of this phase is to analyze and stabilize all the requirements (technical and non-technical), and mitigate any potential high risks in order to derive an architectural foundation that will be sustained until the end of the project.

All architectural decisions are made based on the requirements illustrated through detailed use case models, and only those architecturally significant use cases are used to design the architecture.

The documents and models that are produced from this phase contribute toward the Design Set artifacts, and can include

• An architecture prototype of the system

• Detail use case models

• Analysis model

• Design model

• A development plan for the Construction Phase

• A project plan for the Elaboration Phase

Construction Phase

The objective of this phase is to complete any outstanding analysis work and most of the design and implementation. The software is iteratively and incrementally developed toward a point when the first or beta release of the product can be transitioned into the user community.

The documents and models that are produced from this phase contribute toward the Implementation Set artifacts, and can include

• A Deployment model

• An Implementation model

Transition Phase

During this phase, depending on the development cycle, a beta or final release of the software product is transitioned into the user community. With initial deployments to the user community, bug fixes, addition feature requests, and feature-based training will need to occur.

At the end of this phase, the objectives are measured against the associated requirements, and a decision is made whether to iterate through another development lifecycle.

The Process Workflows of a Rational Unified Process

The Process Workflows outline the steps you actually follow to develop your system through each development cycle of a phase. As illustrated in Figure 2.13, the Process Workflows are

• Business Modeling: Develops an understanding of the problem domain through visual modeling techniques such as use cases.

• Requirements: Use cases are used to functionally specify the system and its boundaries.

• Analysis and Design: A detailed design of how the system will be implemented.

• Implementation: Code development, compilation, and unit testing followed by software deployment.

• Test: Testing the software to ensure it meets the needs of the end users.

• Deployment: Deployment of the software to the actual end user community, and providing any supporting documentation and training.

Figure 2.13 Each of the Process Workflows has associated deliverables.

The supporting workflow processes include Configuration and Change Management, Project Management, and environment.

Even though the activities associated with a specific workflow process can overlap into multiple phases, as illustrated in Figure 2.10, the objectives of the activities will be governed by the phase they are executed within.

Rational Unified Process Resources

To get a better and detailed perspective of RUP, you should visit the Rational Software Web site (http://www.rational.com) or review the following books:

• The Rational Unified Process, An Introduction (The Addison-Wesley Object Technology Series), by Philippe Kruchten

• The Unified Process Explained, by Kendall Scott (Paperback)

• The Road to the Unified Software Development Process (Sigs Reference Library), by Ivar Jacobson and Stefan Bylund

• A Practical Guide to Unified Process, by Donald Kranz and Ronald J. Norman