- 1.1 Chapter Objectives
- 1.2 Context
- 1.3 Decision Model for Software Engineering
- 1.4 Summary
This chapter is from the book
This chapter is from the book
This chapter is from the book
1.3 Decision Model for Software Engineering
For many people, the decision-making process consists of three basic (and usually vague) stages:
- Information goes into a decision process.
- A person-dependent miracle occurs.
- A solution or decision comes out.
Professionals often consider people to be experts or gurus in their fields if they can make good decisions easily. This section gives an overview of a decision model whose purpose is to help software professionals manage their decision processes so that they better understand the solutions they generate.
The PEAK decision model presented in Figure 1-2 does not indicate "what a particular decision should be." It simply implies that stakeholders need to carefully examine their decision processes to help ensure that successful decisions are made. When making decisions, stakeholders should consider the inputs and their relationship to the solution as well as the risk assumed with alternative solutions. The model provides insight into how the inputs and outputs of the decision-making process influence the way in which decisions are made.
)
Figure 1-2 PEAK decision model
The inputs to the decision model are the following elements:
- (P) Problem: What is the issue to be resolved or the problem to be solved? The problem statement should provide a clear representation of what needs to be solved.
- (E) Experience: From prior events, what does the decision maker know or know how to do that might help with this problem? Has the decision maker seen or solved a problem like this one before? How appropriate and accurate is the decision maker's historical information?
- (A) Assumptions: What information is accepted as fact without having evidence? What information about the problem does the decision maker abstract away because the information is not thought to be relevant to the solution?
- (K) Knowledge: What conceptual understanding or factual basis can help the decision maker with the problem? What has the decision maker learned since last dealing with a problem like this? What facts does the decision maker have about the problem? What is the environment that surrounds this problem? For instance, when looking at military problems to be solved, soldiers might ask, "What is the current situation and terrain?"
The outputs from the model are the following elements:
- Solution: What do the stakeholders need in order to solve the problem or to resolve the issue? What alternative solutions are feasible, and why? What are the benefits and cost associated with each alternative? Have the relevant stakeholders discussed the issues concerning the alternative solutions and expressed their preferences? The stakeholders may not know whether a solution is successful until they implement it. The results, whether successful or not, of implementing a decision feed back into the model as part of the decision maker's knowledge and experience. In response to a colleague who asked whether he thought himself to be a failure, Thomas Edison said, "Not at all. Now, I definitely know more than a thousand ways for how NOT to make a light bulb" (Rovira and Trias de Bes 2004, 1).
- Assumed Risk: What is the probability that the solution will not work as envisioned? When a stakeholder makes a decision, the stakeholder assumes some level of risk that the solution or decision will not lead to a successful result. Risk is a consequence of making decisions. What are the risks associated with the alternative solutions? Have the relevant stakeholders discussed these risks with respect to their preferences for the alternatives?
Even if the decision maker is not aware of them, the inputs and outputs of a decision always exist when a decision is being made. To manage their decisions, stakeholders need to identify and manage these inputs and outputs when making decisions. The case studies in the book will describe situations in which most but not necessarily all of the inputs are known. Each case study will present the problem, the assumptions, and many facts. You will bring other knowledge and experience as inputs to the decision model. This book will help you use the inputs and feasible outputs of the model, as they occur in the case studies, to evaluate your decision processes.
So, why do decision makers need a model for evaluating decisions? A model helps decision makers think about the following aspects of their decision processes:
- Information that should be considered in making decisions
- The impact that the inputs to the decision process have on the outputs
- Solutions that are feasible with respect to the nature of the problem
- The assumed risk associated with alternative solutions
- Ways to improve the management of decisions
The remainder of this section provides four examples to illustrate how you can apply the model to the decision process. These examples clarify how to apply the model to evaluate alternative solutions and then to make a decision by selecting the best alternative. The first example, "Software Test Rerun Problem," shows how to identify the inputs and outputs to a decision regarding whether to rerun a software test. The second example, "California Bridge Problem," demonstrates how to apply the model to the more complex problem of building a bridge that can withstand earthquakes, a major issue for the California Department of Transportation. This example demonstrates how newly acquired knowledge feeds back into the decision-making process. The third example, "Unfamiliar Legacy Code Problem," highlights the risk that is assumed when making a decision. The last example, "Data-Processing Problem," clarifies why it is important to understand the nature of the real problem before working on a solution. The diagram shown with each story problem summarizes key inputs and outputs for the decision being made.
Case Study: Software Test Rerun Problem
This example involves a testing problem that software developers often face. The scenario for the problem follows. Before leaving the office, Bob started a software regression test on the computer used for testing software products. He expected the test to complete and the test results to be ready sometime during the night. Bob came into the office the next morning and saw the login prompt on the test computer. He knew either that someone logged out the test account or that the machine rebooted. Bob quickly determined that there had been a power failure and that the machine had rebooted when the power resumed. The test that he had been running did not complete before the power failure, and the complete test results were not available.
Bob examined the incomplete test results and thought they did not provide sufficient verification of the software component being tested that would be needed to integrate this component with other software system components. He reasoned that the incomplete test results probably would not be useful to the software engineers who were depending on them. Bob estimated that running the entire test would take about four hours and that the complete results could be available by the afternoon if he reruns the test this morning. He rationalized that it would be better to have complete test results that are late by half a day than to provide the test stakeholders with incomplete results. Bob decided to rerun the test now because he did not want the delivery of the test results to be a day or more late.
Let's use the PEAK model to evaluate Bob's decision. Figure 1-3 outlines the inputs to and outputs from Bob's decision process. The risk associated with Bob's decision is also an output of his decision-making process.
)
Figure 1-3 Software test rerun problem
Inputs
(P) Problem: The software test failed and yielded incomplete test results. What should Bob do?
(E) Experience: Bob knows the test. (He has run this regression test before. He can estimate the execution time and predict potential causes of failure. Therefore, Bob knows the test can be executed completely by noon if he starts the test now.)
(A) Assumptions:
- The test results are insufficiently complete. (Only complete test results are of sufficient value to the software engineers who will use them.)
- The resources are available during the test period (the next four hours).
- The test will execute to completion without problems. (The problem that occurred last night or any other problem will not occur.)
(K) Knowledge:
- Bob knows what test to run and understands the system well enough to run the test (fact).
- The resources are working sufficiently well to successfully run the test (environment).
Outputs
Solution: Bob did not give the first two alternatives in the following list much consideration and chose the last alternative:
- Do not rerun the test. (Deliver incomplete test results to the software engineers.)
- Rerun the whole test later. (The test results will be a day or more late.)
- Rerun the whole test this morning. (The test results will be approximately half a day late.)
Assumed Risk:
- Incomplete results that are available now may be of value to the software engineers who will use them.
- The test may fail, or the completion of the test results may be delayed for some other reason. (For example, another user may have already planned to use the computer for testing during the day. The execution of another test on the computer may delay or interfere in some way with the execution of the target test. The power outage may occur again.)
You may wonder whether each of the inputs is true. The assumption that "only complete test results are of sufficient value" is particularly suspect. If this assumption is correct, then failure of the test to complete would once again preclude the usefulness of the test results. This constraint would necessitate a more controlled test environment (for example, a battery backup for the test computer) including more extensive monitoring of the test execution. Bob could have assumed that any results up to the point where the power failure occurred might be valid. His actual assumption may have overly constrained the decision so that he did not consider other alternative solutions, such as delivering partially complete test results and rerunning only the incomplete part of the test or delivering partially complete test results and rerunning the entire test later after verifying that the test computer would be available. In addition, he is assuming that the power failure will not occur again when he reruns the test. What if the test is the cause of the power failure? An unproven assumption, such as the problem will not occur again, introduces risk for a decision based upon this assumption. If one of the assumptions is not true, the decision when acted upon may result in failure. Decision makers reduce the risk associated with assumptions by gathering the information needed to convert them to facts (knowledge inputs).
A critical aspect to understand about the decision model is that the inputs and outputs exist even if the decision maker does not acknowledge them. Bob made the implicit assumption that he has time to rerun the regression test this morning. Faced with the failed test situation, Bob did not explicitly, at least not yet, think about what tasks he might already have scheduled to do this morning. By explicitly identifying the inputs, decision makers have the opportunity to correct faulty information that they may otherwise use to make their decisions. The model encourages decision makers to explore the various kinds of information that they have about a problem and to carefully determine feasible solutions. The model also guides decision makers to consider risks that may be associated with alternative solutions or decisions. Faulty assumptions or incorrect knowledge about the problem environment introduces the risk that a chosen solution will not or cannot succeed in solving the problem.
Case Study: California Bridge Problem
The California Department of Transportation, Caltrans, was very proud of the state's freeway infrastructure, which included thousands of bridges. Before the 1991 earthquake, the Caltrans engineers thought that they had the perfect model for freeway bridges in California. They used concrete, oblong, cylinder columns that they thought would withstand an earthquake of magnitude 8.49 on the Richter scale. The 1991 earthquake, of magnitude 6.2, changed the engineers' understanding of earthquake movement. Instead of moving up and down as engineers previously understood, this quake generated a side-to-side motion, which caused the bridge support columns to shear. The two-minute earthquake in 1991 collapsed many bridges in California, including part of the Oakland Bay Bridge.
Engineers studied the new models of earthquake movement and decided that the solution was to combine two materials to create a new "unbreakable" column. This column has the same concrete oblong structure in the middle as before but now has a steel sleeve on the outside. The steel sleeve holds the concrete in place against the shear, while the concrete columns still defend against an up-and-down motion. The problem is that the columns need regular maintenance because the steel sleeves rust. The original concrete columns were basically "maintenance free" (El-Azazy).
This scenario shows that some of the inputs can be based on faulty knowledge. Caltrans wanted a solution to bridge survival in earthquakes that was maintenance free. They based their solution on knowledge and experience that they thought was correct. The new earthquake model forced a new decision with the constraint of required bridge maintenance. The idea that knowledge remains fixed and is correct can create an environment in which the risk associated with decisions based upon this knowledge is unacceptable. As the Caltrans case illustrates, the inputs and outputs to making a decision can and sometimes should change. Figure 1-4 shows the inputs and outputs for the initial decision to build concrete supports.
)
Figure 1-4 California bridge problem
Inputs
(P) Problem: Develop a mechanism for bridge support that is earthquake-proof and low maintenance.
(E) Experience: Concrete structures withstood earthquakes in the past.
(A) Assumptions:
- Engineers have a good understanding of how earthquakes move.
- Large, concrete pillars will withstand earthquakes.
(K) Knowledge:
- In California, concrete is basically maintenance free (fact).
- Concrete structures will withstand forces caused by the up-and-down movement of earthquakes (fact).
Outputs
Solution: Build round, concrete pillars to support bridges.
Assumed Risk: Will bridges supported by concrete pillars survive earthquakes that exhibit other types of motion? (Engineers may not fully understand earthquake movement.)
After the 1991 earthquake, engineers learned that earthquakes exhibit both up-and-down as well as back-and-forth movement. This knowledge led them to formulate a new solution to the problem of building bridges that can withstand earthquakes. The second solution, which involved using steel sleeves and concrete, came with the cost of maintaining the steel supports. The input to the second bridge decision included the new knowledge about earthquake movement as well as the experience of many California bridges collapsing during the 1991 earthquake.
As shown in its application to the California bridge problem, the decision model reflects a "semiclosed" system. There is a feedback loop from the outcome of the first solution into the knowledge and experience inputs for the revised solution. Some inputs are open ended in that they may be influenced but not determined by the outputs. For instance, the outputs from a decision may lead to but not necessarily constrain future problems. Likewise, the output from a prior decision can affect but not control a decision maker's assumptions. Decisions makers control their own assumptions. The output from a decision may impact the environment (knowledge) of a future problem, but it will not be the sole influence because many changing factors affect the environment for a problem.
Case Study: Unfamiliar Legacy Code Problem
This short story helps illustrate how a decision maker's awareness of the risk assumed with alternative solutions may influence the decision that is actually made. The example also clarifies the role of the decision model in helping decision makers brainstorm about alternative solutions to a problem.
Iola is inspecting some "ancient" legacy code and discovers some unfamiliar routines. She asks her colleagues whether they know what the routines do, but most people reply that they don't know. The entire system is too large to rebuild and test as a whole, so she temporarily forgets about the code. But the mysterious code continues to bother her. What if the code performs no useful function and is just wasting space? Maybe she should comment it out or refactor it to be more efficient. She knows that the company is looking for ways to improve legacy code, but removing the code might cause another part of the system to fail or to produce bad results.
Iola decides that the risk of system failure is minimal. Therefore, she decides to comment out the unfamiliar code and to reload the component containing this code into the test environment. She has observed that changes to the other code in the component seem to affect the regression test results, so she assumes that the unfamiliar code will do the same. After three hours of regression testing, Iola detects no failures. She is not sure how to proceed because she has not seen the system respond to the removal of the unfamiliar code. In other words, there has been no smoke from the smoke test. Since the software developers have never built a whole-system regression test, Iola is concerned that she may never be able to know conclusively whether removing the code has a negative impact on the system. Iola decides that the risk of removing the code is too high because she can't determine whether the removal actually has an impact on the system. She uncomments the code and puts it back into the test system. Figure 1-5 shows the inputs and outputs for Iola's initial but not final decision.
)
Figure 1-5 Unfamiliar legacy code problem
Inputs
(P) Problem: What does the legacy code do, and how should it be handled?
(E) Experience: Iola understands what most of the other code does and is able to test it.
(A) Assumptions: The unfamiliar legacy code should be there for a reason.
(K) Knowledge:
- The other code in the legacy component seems to have a purpose (fact).
- Other code changes seem to result in test result changes (fact).
Outputs
Solution: Comment out the unfamiliar legacy code, and see what happens.
Assumed Risk: Removing the unfamiliar legacy code may cause something bad to happen.
Through careful examination of inputs to the model, the problem solver may think of alternative solutions to the problem. Since Iola knows that other code in the component seems to affect the regression tests, maybe she could comment out the unfamiliar code and replace it with a message to indicate that the unfamiliar code would now be executing. She might also add some code to log the environment when the message code executes so that she can trace back to the caller of the routine containing the unfamiliar code.
The decision model is a convenient way to encapsulate factors that influence the result of the decision-making process. A model is more comprehensible than an abstract idea because it is a representation of reality. But since a model is not reality itself, it allows decision makers to examine and evaluate aspects of reality without altering them. Lastly, the model provides a frame of reference for studying information that people use to make decisions.
Case Study: Data-Processing Problem
This last example will show how software project managers and software engineers can use the model to help manage their decisions. This story highlights the challenge of determining the real problem to be solved. If they are not currently overwhelmed, people who enjoy solving problems are curious when they hear that there is a problem. They expect the person with the problem to state very carefully all the facts concerning the problem. In the Star Trek Voyager television show, the character Doctor asks, "Please state the nature of the medical emergency" (CBS Studios). The doctor would not be able to help if the character with the problem responded by giving a life history or a synopsis of the next episode of Days of Our Lives.
Similarly, a software engineer may receive unhelpful information about a problem but still be expected to provide a solution to the person with the problem. The software engineer may not have a clear set of facts from which to start. Organizing the information one needs to solve a problem to make a proper decision can be difficult. The decision model will help problem solvers determine whether they have the information needed to determine appropriate solutions.
In particular, the model helps decision makers begin the process of "divide and conquer" in a systematic way. They start by perusing all the information (data) that they have received to find the four decision-making inputs. First, they ask themselves whether they have sufficient information to clearly identify the problem or issue that needs a solution. If they do, then they work on the other inputs needed for the decision. If they cannot clearly identify the real problem, then they will not be able to determine an appropriate solution.
In this example, an engineer came to the head of an engineering data-processing organization with a request. He wanted the latest and best-performing computer. The manager asked him why he needed the machine. At the time, the cost of this machine was at least seven times that of the standard machine being used within the organization. The engineer said that his processing system was too slow and was therefore causing him to fall behind in his analysis of data. He thought that the advanced machine would be five times faster than his current processor.
The manager knew the machine that the engineer wanted and agreed about its excellent performance, but she asked whether the engineer knew for certain that the processor was the cause of the slowness. The manager then asked the engineer whether the network could be the problem and explained that a faster data-processing machine would not help if the network was the bottleneck. The manager said that she could give the engineer three computers of the standard type today and that it would probably take 30 days for the new, nonstandard system to arrive.
The manager worked with the engineer and the networking people to determine whether the network was the bottleneck. They found that the network was responsible for the slowness in processing data needed for reports. With this new knowledge, the engineer explored ways to use the network more efficiently and discovered that he could improve the throughput of processed data by using multiple, standard machines working on parts of the data in parallel.
The story illustrates that software professionals need to clearly understand their problems as well as the other inputs before applying any decision process. If the inputs to the decision process are invalid, then the output is most likely to be an invalid solution. Software professionals intuitively know this, but because of time and other constraints, they do not necessary keep this in mind when they are solving problems. The decision model helps software professionals separate issues and ensure that they do not end up with "garbage in, garbage out."
Figure 1-6 shows the inputs and outputs for the engineer's initial solution.
)
Figure 1-6 Data-processing problem, initial solution
Before Discussion with the Manager
Initial Inputs
(P) Problem: The engineer's data analysis is falling behind.
(E) Experience: The engineer has not been able to get the data analysis completed when needed for reports.
(A) Assumptions:
- All the data is available to be processed when needed.
- The network is providing the data as needed.
(K) Knowledge:
- The tasks are currently processed in sequence (fact).
- The network transfers data to be processed for reports (fact).
- The engineer sees an advertisement for a computer that is five times faster than the one he currently has (fact).
Initial Outputs
Solution: Buy the new machine that will process data five times faster.
Assumed Risk:
- The software engineer may not receive permission to buy the machine.
- The machine may not solve the engineer's processing problems.
Figure 1-7 shows the inputs and outputs for the engineer's final solution.
)
Figure 1-7 Data-processing problem, final solution
After Discussion with Manager
The engineer made the following changes to his decision inputs after talking with his manager.
Inputs
(P) Problem: The engineer's data analysis is falling behind.
(E) Experience: Multiple machines working in parallel can use network more efficiently.
(A) Assumptions: A number of tasks in the report process can be done in parallel.
(K) Knowledge:
- The network is the bottleneck. (The network was responsible for the slowness in processing data needed for reports.)
- While the network transfers data, it is possible to process report data. (Although many tasks are worked on in sequence, some can be processed in parallel.)
Outputs
Solution: Use a number of in-stock computers to work on parallel parts of the report at the same time.
Assumed Risk: The report processing tasks may not be sufficiently parallel.