- Why Evaluate an Architecture?
- 2 When Can an Architecture Be Evaluated?
- 3 Who's Involved?
- 4 What Result Does an Architecture Evaluation Produce?
- 5 For What Qualities Can We Evaluate an Architecture?
- 6 Why Are Quality Attributes Too Vague for Analysis?
- 7 What Are the Outputs of an Architecture Evaluation?
- 8 What Are the Benefits and Costs of Performing an Architecture Evaluation?
- 9 For Further Reading
- 10 Discussion Questions
This chapter is from the book
This chapter is from the book
This chapter is from the book
2.5 For What Qualities Can We Evaluate an Architecture?
In this section, we say more precisely what suitability means. It isn't quite true that we can tell from looking at an architecture whether the ensuing system will meet all of its quality goals. For one thing, an implementation might diverge from the architectural plan in ways that subvert the quality plans. But for another, architecture does not strictly determine all of a system's qualities.
Usability is a good example. Usability is the measure of a user's ability to utilize a system effectively. Usability is an important quality goal for many systems, but usability is largely a function of the user interface. In modern systems design, particular aspects of the user interface tend to be encapsulated within small areas of the architecture. Getting data to and from the user interface and making it flow around the system so that the necessary work is done to support the user is certainly an architectural issue, as is the ability to change the user interface should that be required. However, many aspects of the user interface—whether the user sees red or blue backgrounds, a radio button or a dialog box—are by and large not architectural since those decisions are generally confined to a limited area of the system.
But other quality attributes lie squarely in the realm of architecture. For instance, the ATAM concentrates on evaluating an architecture for suitability in terms of imbuing a system with the following quality attributes. (Definitions are based on Bass et al. [Bass 98])
Performance: Performance refers to the responsiveness of the system—the time required to respond to stimuli (events) or the number of events processed in some interval of time. Performance qualities are often expressed by the number of transactions per unit time or by the amount of time it takes to complete a transaction with the system. Performance measures are often cited using benchmarks, which are specific transaction sets or workload conditions under which the performance is measured.
Reliability: Reliability is the ability of the system to keep operating over time. Reliability is usually measured by mean time to failure.
Availability: Availability is the proportion of time the system is up and running. It is measured by the length of time between failures as well as how quickly the system is able to resume operation in the event of failure.
Security: Security is a measure of the system's ability to resist unauthorized attempts at usage and denial of service while still providing its services to legitimate users. Security is categorized in terms of the types of threats that might be made to the system.
Modifiability: Modifiability is the ability to make changes to a system quickly and cost effectively. It is measured by using specific changes as benchmarks and recording how expensive those changes are to make.
Portability: Portability is the ability of the system to run under different computing environments. These environments can be hardware, software, or a combination of the two. A system is portable to the extent that all of the assumptions about any particular computing environment are confined to one component (or at worst, a small number of easily changed components). If porting to a new system requires change, then portability is simply a special kind of modifiability.
Functionality: Functionality is the ability of the system to do the work for which it was intended. Performing a task requires that many or most of the system's components work in a coordinated manner to complete the job.
Variability: Variability is how well the architecture can be expanded or modified to produce new architectures that differ in specific, preplanned ways. Variability mechanisms may be run-time (such as negotiating on the fly protocols), compile-time (such as setting compilation parameters to bind certain variables), build-time (such as including or excluding various components or choosing different versions of a component), or code-time mechanisms (such as coding a device driver for a new device). Variability is important when the architecture is going to serve as the foundation for a whole family of related products, as in a product line.
Subsetability: This is the ability to support the production of a subset of the system. While this may seem like an odd property of an architecture, it is actually one of the most useful and most overlooked. Subsetability can spell the difference between being able to deliver nothing when schedules slip versus being able to deliver a substantial part of the product. Subsetability also enables incremental development, a powerful development paradigm in which a minimal system is made to run early on and functions are added to it over time until the whole system is ready. Subsetability is a special kind of variability, mentioned above.
Conceptual integrity: Conceptual integrity is the underlying theme or vision that unifies the design of the system at all levels. The architecture should do similar things in similar ways. Conceptual integrity is exemplified in an architecture that exhibits consistency, has a small number of data and control mechanisms, and uses a small number of patterns throughout to get the job done.
By contrast, the SAAM concentrates on modifiability in its various forms (such as portability, subsetability, and variability) and functionality. The ARID method provides insights about the suitability of a portion of the architecture to be used by developers to complete their tasks.
If some other quality than the ones mentioned above is important to you, the methods still apply. The ATAM, for example, is structured in steps, some of which are dependent upon the quality being investigated, and others of which are not. Early steps of the ATAM allow you to define new quality attributes by explicitly describing the properties of interest. The ATAM can easily accommodate new quality-dependent analysis. When we introduce the method, you'll see where to do this. For now, though, the qualities in the list above form the basis for the methods' capabilities, and they also cover most of what people tend to be concerned about when evaluating an architecture.