- Considerations for an Elephant Eater
- Systems Integration and Engineering Techniques
- Abstraction Is the Heart of Architecture
- Do We Need a Grand Unified Tool?
- The Connoisseur's Guide to Eating Elephants
This chapter is from the book
This chapter is from the book
This chapter is from the book
Systems Integration and Engineering Techniques
But the problem we're talking about isn't new, is it? People have been trying to deliver complex systems for more than 40 years. There must already be some pretty reasonable Elephant Eaters out there.
Now that we have a good understanding of the problem, it's a good idea to take a closer look at some of the solutions that are already out there and see why, given the meager 30 percent success rate noted in the Preface, we need a new Elephant Eater.
Generally, these big problems need to be approached via formal techniques. These techniques work from two different directions. They either work their way down from the top, gaining increasing levels of detail, or they start from the bottom, examining needs in detail and working their way upward, building toward a complete solution.
Walk the Easy Path or...
If you're infinitely lucky, the bottom-up approach might work. Considering a very simple example, you could select a package that seems close to what you need. You could then walk through the business processes you want to execute. As you go, you can write down all the changes you need to make to the package, and, presto! After you've made the changes, you've got a solution! You've designed the whole system from the ground up because the package dictates your choices for how you do pretty much everything else.
If you don't allow the package to dictate your choices, chances are, you will find yourself in a very sticky mess: Each major change you make will require extra development, testing, and long-term maintenance costs. If you've chosen the bottom-up approach, you must stick to it religiously and accept the changes it will impose on the process and the business.
Ultimately, a package with a good fit, whether imposed or a lucky choice, is the very best in bottom-up solutions. Start halfway up the hill—the package already approximates what you want. Then modify the solution iteratively with the end user and find a happy endpoint near the top of the hill.
However, chances are, for a really complex project, using the bottom-up approach with a single package will not work. You must break down the problem into smaller pieces and then integrate them to create a single solution. You can divide up the problem in two fundamental ways.
...Break the Boulders and Make Them Smooth
You can decompose the problem into smaller, more easily managed Views through two methods: splitting and abstraction. Splitting simply divides complex big chunks into smaller, more manageable pieces. Abstraction removes detail from each larger chunk to form more manageable and understandable pieces. These two techniques, splitting and abstraction, allow almost any gargantuan problem to be subdivided into smaller, better contained problems. Think of it as slicing the problem into little squares.
Abstraction gives you horizontal cuts, while View splitting gives you vertical ones. Everything becomes a manageable "chunk." This is the basis for most systems integration and engineering methods. Many of these methods are proprietary, but some, such as The Open Group Architecture Framework (TOGAF) from the Open Foundation, are freely available. Each approach tries to create a continuum of knowledge, from high-level representations to more detailed. These paths vary but can be characterized as moving in some way from logical to physical, general to specific, or taxonomy to specification.
When good methods or tools are used, there is traceability from the high level to the low level. This helps a reader understand why something has been designed the way it has.
Such movement is unsurprisingly characterized as a progression, starting from the high-level principles and overall vision of what needs to be achieved, and moving down through the perspectives of business, process, roles, and models of information. Figure 6.1 highlights the basic stages of the TOGAF method.
)
Figure 6.1 Even with its cyclic diagram, TOGAF is part of the progressive school of architecture.
Some approaches go even further. They segment each level of abstraction into a number of separate perspectives. Of these "frameworks," the enterprise architecture framework produced by John Zachman of IBM in the 1980s is probably the most famous. Called the Zachman Framework, it considers the additional dimension of Data, Function, Network, People, Time, and Motivation. Figure 6.2 illustrates how the Zachman Framework segments the architecture into these perspectives.
)
Figure 6.2 The Zachman Framework of Enterprise Architecture segments the architecture into a variety of perspectives.
These approaches enable you to decompose the full width and breadth of the problem (including the existing constraints) into separate Views so that a suitably skilled guru can independently govern and maintain them.
At the very top of this top-down approach is a simple sheet of paper that purports to show or describe the scope of the whole problem for that particular perspective. A single sheet of paper might even purport to summarize the 10,000-foot view for all the perspectives.
Below that top sheet are many more sheets that describe each element on the sheet above. This technique is so well recognized that it's applied to almost everything in complex problems, whether we're talking about the shape of the system, the business processes that it executes, or the description of the plan that will build it.
In this hierarchy of paper, the top tier is labeled Level 0; the next tier down, Level 1; and so on. At each layer, the number of sheets of paper increases, but each of these sheets is a manageable View. The problem has been successfully decomposed. In the example in Figure 6.3, our single-page business context that describes the boundaries of the problem we're solving is gradually decomposed into 60,000 pages of code, deployment information, and operational instructions that describe the whole solution. At each step of the way, the intermediate representations all correspond to a View.
)
Figure 6.3 Decomposition of a complex problem space
After the problem has been decomposed into single sheets, or Views, rules must be written and applied to specify how they work together.
Surely that solves our problem. The elephant has been eaten. Complexity is reduced, so each area becomes manageable. Each person is dealing with only a bit of the problem.
This is, of course, precisely what the world's largest systems integrators do. They define their Views in terms of work products or deliverables. They come from different perspectives and at different levels of abstraction. The systems integrators have design and development methods that describe who should do what to which View (or work product) and in what order.
So if the problem is essentially solved, why does it go wrong so often?