This chapter is from the book
This chapter is from the book
This chapter is from the book
What Is Design?
Design is so familiar that few people give it any thought, but the more you think about design in general, the more peculiar the design of IT applications is shown to be. This peculiarity provokes a question: Is application design really so different, or do we go about it in such an odd way that we are obscuring some fundamental truths?
To begin with, let us define design. A design is a conceptualization of the thing or system you wish to build. The word design is both a noun and a verb, but the definition in the previous sentence is for the noun. So we supplement it by saying that the verb design means to create a design (the noun).
This definition is broad, deliberately so. Even if you haven’t drawn a nice diagram of the imagined final product or written a textual description of the thing or system, even if the conceptualization consists only of a thought in your head, it is still a design.
Of course, people do design in many different ways, but when you take design down to its essential structure, there is a deep similarity among all kinds of design. I think most designers understand this intuitively, though I have to admit I have never seen it articulated like this (though I once saw it articulated somewhat like this but in a manner that was so opaque I’m not absolutely sure).
The structure of design is as follows. Design is a three- or four-step process:
- Understanding. Understand what you are trying to achieve—people’s ambitions for the development—and the problems that you will need to overcome. The outcome is the list of requirements.
- Hypothesis. This is the creative bit; this is where hunches and informed guesswork come in. Have some ideas and choose one (or maybe more) to carry forward.
- Elaboration. Flesh out the design by making sure you have covered all the requirements and have fixed all identified weaknesses.
- Analysis. Perform checks, tests, and/or calculations on the design. Try to kill the hypothesis.
It is the fourth step that is optional—or at least it is often not done.
An example is designing a bridge. The requirement is simply to create a way X vehicles per hour can cross a river at point Y, give or take half a mile each way, at no more than Z cost. A tunnel is ruled out on cost grounds. The design hypothesis would be whether to build a suspension bridge, a box girder bridge, a Japanese traditional wooden-style bridge, a typical stone arched bridge, or one of the many other types of bridges. The elaboration would determine what it looks like, where the columns are placed, what it is made of—the details, in other words. The analysis would be the calculations to see whether it could carry the load and won’t fall down if the river is flowing fast and other checks.
While I have described these as steps, in practice the process is rarely so clear-cut. In particular, analysis may be spread throughout the elaboration step, although it is important to redo the analysis when you think the design is finished. Even choosing a design hypothesis may be immediately followed by a quick analysis check to establish that the idea is sound. Part of elaboration is driven by looking for weaknesses in the design and finding ways to fix them. Analysis checks are one tool for looking for weaknesses.
In IT, a popular notion is the idea of patterns. Patterns are rather like prepared design hypotheses. The idea is an old one. In the recent past it was associated with the architect Christopher Alexander, but in the eighteenth century there were many pattern books published to help less well-trained builders create classical-style houses. Patterns are good, but there is a point where designers must be more creative and think afresh to find a solution.
In design there is feedback, meaning that you might have to loop back to an earlier step. During the elaboration step, you might find that the design hypothesis doesn’t work and you have to go back to try another approach, in other words, select another hypothesis. Also—and this is important—you may find that during the elaboration step you have to add to or change your requirements because you uncover gaps or inconsistencies. When the hypothesis changes, the requirements focus changes. For instance, if you are designing a bridge and you decide to change the design hypothesis from a suspension bridge to a bridge that lifts or moves the roadway (like Tower Bridge in London or the swiveling bridge at Wuhan), a whole new set of requirements come into play, like how often it raises or swivels. Thus in design even the requirements aren’t fixed. This is illustrated in Figure 1.1.
)
Figure 1.1 Design Feedback
The analysis step may, of course, reveal an error. Sometimes you can fix the error by tinkering with a detail, in other words, by redoing part of the elaboration step. Sometimes you have to go all the way back to step 2—that is, look for another hypothesis.
One of the requirements is likely to be cost. During the elaboration or analysis steps you may realize the cost is too great and the whole design must be reconsidered.
The word hypothesis points at the similarities between design and advancing a scientific theory. This is deliberate. The notion of a hypothesis and trying to disprove it is associated with Karl Popper and is now more or less accepted as how we do science. Before Popper advanced his hypothesis about hypothesizing, it was thought that scientific progress took place by accumulating facts until a scientific law emerged. This model of accumulation and emergence is similar to how most people think of design, and I think this is how most people think of application development; we gather requirements and the design emerges. It is as wrong for application design today as it was wrong for science in the past, for two main reasons. First, we are not dealing in certainties; we are dealing with guesswork, intuition, and gut feeling—there is no one answer. The elaboration and analysis steps are about keeping the guesswork, intuition, and gut feelings in check, putting them on a sound basis. The second reason is that, as previously noted, the hypothesis and the elaboration may change the requirements. There is a kind of conversation between the requirements and the solution that needs to take place.
Because of the feedback between the steps, design does not work well in a fixed time frame. All we can hope for is that the duration of doing the design is much less than the duration of the implementation. This is a problem for IT design because programming is itself a design exercise. The aim of much of this book is to look for ways of making important decisions early (but not too early) so that the programming is relatively routine.
I believe the best designers go through these four steps instinctively. They have two behaviors that poor designers don’t have. First, they avoid many of the problems of looping back by keeping in their heads a picture of the design that is way ahead of what they are actually doing. Put another way, they loop back but in their heads. Second, they take their time on major decisions like choosing the hypothesis; they don’t leap to conclusions, and they are happy to listen to suggestions and explore alternatives. At various points in this book I urge you to think like a designer; what I mean by this is that I would like you to think deeply about the hypothesis and what it means for the elaboration step and about the dialogue between design and requirements. You should also be looking for ways to analyze the design.
Design is done in many different ways. I group these broadly into ad hoc design, planned design, and engineered design. These are discussed in the following sections.
Ad Hoc Design
In ad hoc design you start with a simple sketch or just an idea and start building. As you go along, you are likely to find that the design is incomplete or has problems, so you tweak it. Sometimes you have to make changes to the design that are so fundamental that you have to undo some of the completed work.
Ad hoc design is extremely common in IT, and I have used it myself a lot. It works well when you are building something that is routine—you have done something similar before and more or less know what to expect. More precisely, it normally works fine if the project has these characteristics:
- The project is small. The design is in one person’s head and therefore the designer is able to set very well-defined jobs for anyone else on the team. Alternatively, the designer can subcontract part of the design to another member of the project, but again only if the scope and the requirements for the subdesign can be set precisely.
- The project is self-contained, by which I mean that other development projects happening at the same time aren’t dependent on it. In general, if a project depends on another IT project, it must define precisely the nature of the dependency. For instance, it must specify exactly what data it receives and what data it sends or what data it shares in a database. This is hard to do in an ad hoc design project because by the nature of the process the designer is likely to change his or her mind, which may infuriate the designers of the dependent project.
- The relationship between the designer and the stakeholders is very close because the stakeholders have to trust the designer to deliver what they want with very little evidence. This problem can be alleviated by frequently showing the stakeholders the partially completed build, which is fine if the stakeholders have the time and energy continually to review the work in progress.
There is another scenario where ad hoc design works, and that is when you start on a project and don’t know where you are going to end up. Essentially it becomes design by trying things out. An example I use in this book is an application I built for drawing diagrams from data. I used ad hoc design partly because the stakeholder was me, and partly because when I started I didn’t have a clear idea of what the final product would look like.
Ad hoc design works best if you are mentally prepared at the beginning to throw away work you have already done. This happens because as you go along the design road, you sometimes reach a roadblock or you realize there is a better solution to the problem. I reckon that in my diagram-drawing application I threw away about 20% of already-tested code I had written. Artistic work can also be seen as a kind of design, and it is often achieved using ad hoc design. A characteristic of great artists is their willingness to destroy work that isn’t good enough. I would go as far as to say that you can design a solution to a complex and challenging problem with ad hoc design but only if you are prepared to throw away large parts of the unfinished product. Put simply, you code something, realize you can do better, and code it again.
It is commonly observed that the second or third release of a software product is much better than the first release. Clearly the interrelationships among parts of the system are much better understood if a version of the system already exists, and thus it is easier for different designers to be working on different parts of the system in harmony. And it’s partly a marketing thing; it is better to have something working to gain entry to the market, so the first version might be a bit sloppy. (That said, I have never known good programmers to write sloppy code even if they are under time pressure because it is usually counterproductive.) But I suggest a major reason is that when writing the second or third release, people are much more willing to throw away code belonging to the old release, which they wouldn’t have done while programming the first release.
One way to create a large product using ad hoc design is to have many small releases. A complex design builds up gradually, and as it emerges it is understood by all members of the implementation team. As a tangential point, observe that something very odd is happening here. Why is it we can understand a large, complex piece of software in our heads, but it seems we can’t express the design on paper in such a way that people can understand it? Much of this book is about my attempt to do this.
Planned Design
In planned design you draw a diagram of the product you are designing or describe it in words. Most of the fine historical buildings you see today were designed using planned design; an architect drew a diagram of the building’s floor plan and exterior walls and specified what materials and/or techniques to use. I expect most of the wonderful Gothic cathedrals in Europe were designed using planned design; it is hard to imagine how the builders could have done it any other way, especially when you consider someone carving a block of stone to fit exactly into one position in the window tracery. But Gothic cathedral builders used rules of thumb to decide how thick to build the columns, and they relied on craftspeople to know how to build what was in the drawing.
A great benefit of planned design is that, assuming you are using competent craftspeople, you don’t have to supervise them as much as in ad hoc design. They can look at the drawing and build from that.
When you have something complicated to build, design hierarchies come into play. Suppose I am designing a medieval cathedral. At the top level of design is the basic shape. This defines where the pillars and the walls go but not the internal structure of the pillars or roof. This detailed design is left to others, or more likely to craftspeople who will build it according to the same designs they have used for centuries. Someone else might design the stone tracery, the stained-glass windows, and the layout of the altar and choir stalls.
Planned design tends to be conservative because it relies on a body of knowledge and skills in the craftsperson’s head, whereas moving away from the familiar requires developing a new body of knowledge and skills.
IT has used planned design a great deal, especially in large projects. I think it is fair to say that planned design has a mixed record of success in application design.
Engineered Design
Engineered design is very much like planned design in that the design is represented by drawings or text; the difference is that in engineered design the design is tested. In other words, engineered design is the only kind of design that includes the formal analysis step previously described in the four-step process. I am not saying that in planned design or ad hoc design there is no analysis, because I suspect good designers often do some analysis in their heads even if it is something as simple as running through different scenarios of how the product will be used. I even suspect that Gothic cathedral designers in the Middle Ages thought long and hard about how the cathedral would be used, perhaps even working out how the space could give added flexibility to processions and services. Unfortunately the cathedral builders did not understand forces and load-bearing calculations, and quite a number of cathedral towers fell down shortly after they were built. The difference in engineered design is that formal analysis is done and there is a body of knowledge about what analysis is appropriate. A modern civil engineer will calculate the loads and will have confidence that, so long as the build is done correctly, the building will be structurally sound. There are other forms of analysis besides calculation. You can analyze a building design to check how long it will take people to escape if there is a fire by using computer modeling. You can analyze a car design for wind resistance by putting it in a wind tunnel.
For engineered design of anything beyond the very simple, there will be a hierarchy of designs. For instance, if you are designing a new passenger jet plane, the top-level design describes the shape of the plane in terms of how all the components fit together. Each component has its own design, and a complex component like an engine is also broken down into numerous subcomponents, each with its own design. The hierarchical nature of the design is illustrated rather abstractly in Figure 1.2.
)
Figure 1.2 Hierarchical Design of an Aircraft
Each small box is a component, and each big box enclosing the small boxes is a design. Thus the top-level design shows how the aircraft is an assembly of body, wings, engines, undercarriage, and so forth, and the engine design shows how the engine parts fit together.
The requirements for the component design come both from the higher-level designs and from the outside. For instance, an aircraft engine’s requirements come from the higher-level design—power needed, weight limits, size limits—and from the outside—noise limits and serviceability requirements. This is illustrated in Figure 1.3.
)
Figure 1.3 Requirements on a Component
In engineered design each component design is tested.
There are two important points about engineering calculations. The first is that the calculations depend on the components satisfying their requirements. For instance, an aircraft designer will do a calculation to check whether the plane will fly. This calculation relies on assumptions about how much the engines weigh and how much power they deliver. Any factor of a component used in the analysis must be underpinned by a requirement on the component design. If the engines cannot deliver the power for the prescribed weight, the higher-level design must be redone. In other words, there is a feedback loop from component design to the higher-level design. This is illustrated in Figure 1.4.
)
Figure 1.4 Redesign Feedback Loop
The second important point about engineering calculations is that they do not prove that something will always succeed. All they do is show that the product won’t fail under a set of predefined circumstances. Reverting to the building example, structural engineers can calculate loads to be sure that a building won’t collapse when people walk in and furniture is added. But to be sure the building won’t fall down in a high wind you have to do another set of calculations. Likewise for earthquakes, floods, and so on. In other words, calculations prove only that problems you have identified are prevented. It does not prove that problems you haven’t identified are prevented. I make this point because in IT there is this nirvana that people hanker for, which is that a program is provably correct. Engineers don’t prove that a design is correct; instead, they prove that it does not suffer from a predefined set of problems. It is time we stop trying to be mathematicians and concentrate on being engineers instead.
The point that failure has to be anticipated before it can be fixed is illustrated by the famous example of the Tacoma Narrows Bridge collapse in 1940 2. What happened was that a medium-strength wind caused resonance, and the swaying built on itself. The consequence was that a small up-and-down movement became a large up-and-down movement, and the bridge eventually writhed about until it broke. These days engineers understand resonance and do calculations to ensure that a structure does not suffer from it. Sometimes they get it wrong, as with the Millennium Bridge in London. This had resonance, not from wind but from people walking over it. The bridge was closed shortly after opening while the problem was fixed.
There is always a danger that some problem is found that cannot be surmounted without a redesign. And there is always a danger that a redesign of a component will cascade, leading to a further redesign of the whole product. Both dangers are likely to be more serious in planned design than in engineered design because in engineered design the analysis is likely to catch problems early.
Summary of Design Approaches
There were a number of concepts in the last few sections, and I think it is worth a recap.
- Design is essentially a four-step process: understanding, hypothesis, elaboration, and analysis.
- Complex designs can be split into a hierarchy of components being separately designed.
Broadly speaking, there are three kinds of design:
- Ad hoc design. There is no formal design, which means the design resides in the designer’s head. It works well for small, well-understood products but can also be used for exploratory design if the designer is willing to rework large parts of the product that were previously thought to be finished.
- Planned design. There is a formal design that allows the build to proceed without continual guidance from the designer. It can be used for any scale of project (even large—think of the pyramids!). There is no formal analysis; therefore, the performance of the product is reliant on the designer’s intuition.
- Engineered design. This is like planned design, but formal analysis is applied to the design. Building products that are complex, that have many internal dependencies, or whose performance is hard to predict demands engineered design.
To support the arguments in the next two sections, I have drawn in Figure 1.5 a diagram representing engineered design.
)
Figure 1.5 Engineered Design
So can we make application development an engineering discipline, and if we can, do we want to?