Eating the IT Elephant: Abstraction Works Only in a Perfect World
This chapter is from the book
This chapter is from the book
This chapter is from the book
- "There is no abstract art. You must always start with something. Afterward you can remove all traces of reality."
- —Pablo Picasso
Chapter Contents
- Considerations for an Elephant Eater
- Systems Integration and Engineering Techniques
- Abstraction Is the Heart of Architecture
- Do We Need a Grand Unified Tool?
- The Connoisseur's Guide to Eating Elephants
- Endnotes
In the first part of the book, we saw how IT systems have grown increasingly larger and more complex over time. This growing complexity is challenging the capability of businesses to innovate as more of the IT budget is channeled into regulatory compliance, replatforming, and maintenance of the status quo. As this book has shown, changing these systems is not primarily a technical difficulty, but one of coordinating and disambiguating human communication. In overcoming such difficulties, we have introduced the concept of an Elephant Eater and the Brownfield development approach.
This second part of the book explains the technical and practical aspects of Brownfield for someone who might want to implement such an approach. This chapter examines the necessary technical context, requirements, and characteristics of the Elephant Eater. The chapter then goes on to analyze existing IT elephant-eating approaches and highlights the problems these approaches present with their extensive use of decomposition and abstraction.
Considerations for an Elephant Eater
The following sections outline considerations for the Elephant Eater. The problems with large scale developments are many, and the first half of the book illustrated some of the problems that such developments pose. The high failure rate for such projects is the reason why the creation of an Elephant Eater was necessary. Like any problem, the starting point for a solution is the understanding of the requirements, so if an Elephant Eater is going to be created, it needs to cater to the considerations in this section.
Lack of Transparency
On very large-scale developments, the problem being solved usually is unclear. At a high level, the design and development task might seem to be understood—for example, "build a family home," "design a hospital," or "implement a customer relationship management system." However, such terms are insufficient to describe what is actually required.
For any complex problem, some degree of analysis and investigation is essential to properly frame the detailed requirements of the solution and understand its context. In conventional building architectures, the site survey is a fundamental part of the requirements-gathering process.
A thorough analysis of a complex site takes a great deal of time and effort. Even using traditional Greenfield methods, the analysis effort is often as large as the build effort. Despite this effort, however, IT architects and business analysts rarely do as thorough a job of surveying a site as building architects do. As discussed in previous chapters, a thorough analysis that encompasses functional and nonfunctional requirements and multiple constraints requires vast documentation. As such, the real requirements in any situation are always less than transparent.
Unfortunately, in IT, relatively little time is spent on the equivalent of a site survey.
Multiple Conflicting Goals
Another problem is conflicting requirements. In any complex situation, a single optimal solution is rarely a given for such a problem. The problem itself might even be poorly described.
In the example of the house building discussion in Chapter 1, "Eating Elephants Is Difficult," the mother-in-law and the landowner could have very different perspectives on what is desirable. Will their combined requirements be entirely coherent and compatible? Whose job will it be to resolve these conflicts?
We have seen the same problem on multiple $100 million programs. Any big program owned by more than one powerful stakeholder is likely to fail because of confusing and conflicted directions. As we saw in Chapter 1, life is much easier when one powerful person is consistently in charge. Of course, assigning a single stakeholder is not easy, but failing to identify this stakeholder at the start of the project only ignores the problem.
Spotting requirements that are clearly expressed but in conflict is reasonably easy, and it is usually possible to resolve these through careful negotiation. No one would seriously demand two mutually incompatible set of requirements, right?
Let's return to the analogy of home building as an example. When designing a house, increasing the size of the windows will increase the feeling of light and space within the building and improve the view. But bigger windows will contribute to energy loss. Improved insulation in the walls or ceilings might compensate for this, but this could result in increased building costs or a reduced living area. Alternatively, the architect could request special triple glazing. That would make the windows more thermally efficient but could make the glass less translucent. As more concerns arise, the interplays between them become more complex. As a result, the final solution becomes a trade-off between different aspects or characteristics of the solution. Possibly, the requirements are actually mutually incompatible—but this can be known only in the context of a solution.
These conflicting requirements also come up repeatedly when designing large computer systems. We hear comments similar to these: "We need the system to be hugely scaleable to cope with any unexpected demand. It must be available 24 hours a day, 7 days a week—even during upgrades or maintenance—but must be cheaper to build, run, and maintain than the last system." Obviously, such requirements are always in conflict.
Dynamic Aspects
The difficulty in coping with these requirements is compounded by the fact that they don't stand still. As you begin to interfere and interact with the problem, you change it. For example, talking to a user about what the system currently does could change that user's perception about what it needs to do. Interacting with the system during acceptance testing might overturn those initial perceptions again. Introducing the supposed solution into the environment could cause additional difficulties.
In IT systems, these side effects often result from a lack of understanding that installing a new IT system changes its surroundings. Subsequent changes also might need to be made to existing business procedures and best practices that are not directly part of the solution. These changes might alter people's jobs, their interaction with customers, or the skills they require.
In addition to these impacts, the time involved in such projects usually means that the business environment in which the solution is to be placed has evolved. Some of the original requirements might need to change or might no longer be applicable
Therefore, one of the key requirements for any Elephant Eater is tight and dynamic linkage between the business and IT.