Defining Software Quality and Economic Value

This chapter is from the book

Economics of Software Quality, The

Learn More Buy

This chapter is from the book

This chapter is from the book 

Economics of Software Quality, The

Learn More Buy

Introduction

This book deals with two topics that have been ambiguous and difficult to pin down for many years: software quality and economic value.

The reason for the ambiguity, as noted in the Preface, is that there are many different points of view, and each point of view has a different interpretation of the terms. For example, software quality does not mean the same thing to a customer as it does to a developer. Economic value has a different meaning to vendors than it has to consumers. For vendors, revenue is the key element of value, and for consumers, operational factors represent primary value. Both of these are discussed later in the book.

By examining a wide spectrum of views and extracting the essential points from each view, the authors hope that workable definitions can be established that are comparatively unambiguous.

Software quality, as covered in this book, goes well beyond functional quality (the sort of thing to which customers might react to in addition to usability and reliable performance). Quality certainly covers these aspects but extends further to nonfunctional quality (how well the software does what it is meant to do) and to structural quality (how well it can continue to serve business needs as they evolve and change as business conditions do).

Why Is Software Quality Important?

Computer usage in industrial countries starts at or before age 6, and by age 16 almost 60% of young people in the United States have at least a working knowledge of computers and software. Several skilled hackers have been apprehended who were only 16 years of age.

The approximate population of the United States in 2010 was about 309,800,135 based on Census Bureau estimates. Out of the total population about 30% use computers daily either for business purposes or for recreational purposes or both; that is, about 92,940,040 Americans are daily computer users.

About 65% of the U.S. population use embedded software in the form of smart phones, digital cameras, digital watches, automobile brakes and engine controls, home appliances, and entertainment devices. Many people are not aware that embedded software controls such devices, but it does. In other words, about 201,370,087 U.S. citizens own and use devices that contain embedded software.

Almost 100% of the U.S. population has personal data stored in various online databases maintained by the Census Bureau, the Internal Revenue Service, state governments, municipal governments, banks, insurance companies, credit card companies, and credit scoring companies.

Moving on to business, data from various sources such as Forbes, Manta, Business Week, the Department of Commerce Bureau of Labor Statistics, and others reports that the United States has about 22,553,779 companies (as of the end of 2010). Of these companies about 65% use computers and software for business operations, retail sales, accounting, and other purposes—so about 14,659,956 U.S. companies use computers and software. (Corporate software usage ranges from a basic spreadsheet up to entire enterprise resource planning [ERP] packages plus hundreds of other applications.)

Based on data from the Manta website, the software deployed in the United States is provided by about 77,186 software companies and another 10,000 U.S. companies that create devices with embedded software. A great deal of embedded software and the device companies themselves have moved to China, Taiwan, Japan, India, and other offshore countries. An exception to offshore migration is the manufacture of embedded software for military equipment and weapons systems, which tends to stay in the United States for security reasons.

The U.S. military services and the Department of Defense (DoD) own and deploy more software than any other organizations in history. In fact, the DoD probably owns and deploys more software than the military organizations of all other countries combined. Our entire defense community is now dependent on software for command and control, logistics support, and the actual operation of weapons systems. Our national defense systems are highly computerized, so software quality is a critical component of the U.S. defense strategy.

Without even knowing it, we are awash in a sea of software that operates most of our manufacturing equipment, keeps records on virtually all citizens, and operates the majority of our automobiles, home appliances, and entertainment devices. Our transportation systems, medical systems, and government operations all depend on computers and software and hence also depend on high software quality levels.

While software is among the most widely used products in human history, it also has one of the highest failure rates of any product in human history due primarily to poor quality.

Based on observations among the authors' clients plus observations during expert witness assignments, the cancellation rate for applications in the 10,000 function point size range is about 31%. The average cost for these cancelled projects is about $35,000,000. By contrast, projects in the 10,000 function point size range that are successfully completed and have high quality levels only cost about $20,000,000.

When projects developed by outsource vendors are cancelled and clients sue for breach of contract, the average cost of litigation is about $5,000,000 for the plaintiff and $7,000,000 for the defendant. If the defendants lose, then awards for damages can top $25,000,000. However because most U.S. courts bar suits for consequential damages, the actual losses by the defendants can be much larger.

Of the authors' clients who are involved with outsourcing, about 5% of agreements tend to end up in court for breach of contract. The claims by the plaintiffs include outright failure, delivery of inoperable software, or delivery of software with such high defect volumes that usage is harmful rather than useful.

As of 2011, the average cost per function point in the United States is about $1,000 to build software applications and another $1,000 to maintain and support them for five years: $2,000 per function point in total. For projects that use effective combinations of defect prevention and defect removal activities and achieve high quality levels, average development costs are only about $700 per function point and maintenance, and support costs drop to about $500 per function point: $1,200 per function point in total.

Expressed another way, the software engineering population of the United States is currently around 2,400,000 when software engineers and related occupations such as systems analysis are considered. On any given day, due to poor quality control, about 1,000,000 of these workers spend the day finding and fixing bugs (and, unwittingly, injecting new bugs as part of the process).

So all of these statistics point to the fact that better software quality control in the forms of defect prevention and more effective defect removal could free up about 720,000 software personnel for more productive work than just bug repairs, easily reducing U.S. software development and maintenance costs by about 50%.

As we show later in the book, the cost savings that result from higher quality are proportional to application size. As software projects grow larger, cost savings from high quality levels increase. Table 1.1 illustrates typical software development costs for low, average, and high-quality software applications.

Table 1.1. Software Costs by Size and Quality Level

The technologies and methods associated with these three quality levels are discussed and illustrated in later sections of this chapter, as are the reasons that large software projects are so risky. Suffice it to say the "high quality" column includes effective defect prevention, effective pretest defect removal such as inspections and static analysis, and much more effective testing than the other columns.

Another major reason that software quality is important is because poor quality can and will affect each citizen personally in unpleasant ways. Every time there is a billing error, every time taxes are miscalculated, every time credit ratings change for incorrect reasons, poor software quality is part of the problem.

Early in 2010, hundreds of computers were shut down and many businesses including hospitals were disrupted when the MacAfee antivirus application mistakenly identified part of Microsoft Windows as a virus and stopped it from loading.

According to the July 25, 2010, issue of Computerworld, the BP drilling platform that exploded and sank had been having frequent and serious computer problems for a month prior to the final disaster. These problems prevented significant quantities of data from being analyzed that might have warned operators in time to shut down the oil pumping operation.

If your automobile braking system does not operate correctly, if a home appliance fails unexpectedly, or if a hospital makes a medical mistake, there is a good chance that poor software quality was part of the problem.

If an airline flight is delayed more than about two hours or if there is a widespread power outage that affects an entire geographic region such as New England, the odds, again, are good that poor software quality was part of the problem.

Because software is such a basic commodity as of 2011, it is useful to start by considering how much software ordinary U.S. citizens own and use. Table 1.2 shows typical software volumes associated with normal living activities.

Table 1.2. Personal Software Circa 2011

The data in Table 1.2 comes from a combination of web sources and proprietary data provided by clients who build appliances of various kinds.

Not every citizen has all of these appliances and devices, but about half of us do. Many of us have even more than what Table 1.2 indicates, such as owning several automobiles, several cell phones, and numerous appliances. Software quality is important because it is the main operating component of almost all complex machines as of 2011.

Another reason that software quality is important is because many of us need high-quality software to go about our daily jobs. Table 1.3 shows typical software usage patterns for a sample of positions that include knowledge work, based on observations and discussions with members of various professions and from studies with the companies that provide the software.

Table 1.3. Occupation Group Software Usage Circa 2011

As can be seen from Table 1.3, all knowledge workers in the modern world are heavily dependent on computers and software to perform their jobs. Therefore, these same workers are heavily dependent on high software quality levels. Every time there is a computer failure or a software failure, many knowledge workers will have to stop their jobs until repairs are made. Indeed, power failures can stop work in today's world.

One of the authors was once an expert witness in a software breach-of-contract lawsuit. While being deposed in Boston there was a power failure, and the court stenographer could not record the transcript. As a result, four attorneys, the stenographer, and two expert witnesses spent about two hours waiting until the deposition could continue. All of us were being paid our regular rates during the outage. We are so dependent on computers and software that work stops cold when the equipment is unavailable.

Similar occurrences take place after hurricanes and natural disasters that shut down power. Many retail establishments are unable to record sales information, and some stay closed even though workers and potential customers are both available. If computers and software are out of service, many businesses can no longer operate.

Software and computers are so deeply enmeshed in modern business and government operations that the global economy is at serious risk. As military planners know, nuclear explosions in the atmosphere emit an electromagnetic pulse (EMP) that damages transistors and electrical circuits. They can also cause explosions of liquid fuels such as gasoline and can detonate stored weapons.

Such "ebombs" can be designed and detonated high enough so that they don't cause injuries or death to people, but instead cause major destruction of electronic devices such as radar, electric power generation, television, computers, and the like.

As of 2011, it is thought that most major countries already have ebombs in their arsenals. CBS news reported that one or more ebombs shut down the electric capacity of Baghdad without doing physical damage to buildings or personnel during the second Iraq war. This could be one of the reasons why restoring power to Baghdad after the hostilities ended has been so difficult.

A final reason that software quality is important is because dozens of government agencies and thousands of companies have personal information about us stored in their computers. Therefore, both quality and security are critical topics in 2011.

Table 1.4 shows examples of the kinds of organizations that record personal information and the probable number of people who work in those organizations who might have access to data about our finances, our Social Security numbers, our health-care records, our dates of birth, our jobs, our families, our incomes, and many other personal topics.

Table 1.4. Estimated Applications with Personal Data

Given the number of government agencies and corporations that record vital data about citizens, and the number of people who have access to that data, it is no wonder that identity theft is likely to hit about 15% of U.S. citizens within the next five years.

A Congressional report showed that the number of U.S. cyber attacks increased from about 43,000 in 2008 to more than 80,000 in 2009. As this book is being written, probably more than 10,000 U.S. hackers are actively engaged in attempting to steal credit card and financial information. Computers, networks, and smart phones are all at considerable risk. Security vulnerabilities are linked closely to poor quality, and many attacks are based on known quality flaws.

Because computers and software are now the main tools that operate industry and government, software quality and software security are among the most important topics of the modern world. Indeed, the importance of both quality and security will increase over the next decade.

From an economic standpoint, higher software quality levels can shorten development schedules, lower development and maintenance costs, improve customer satisfaction, improve team morale, and improve the status of the software engineering profession all at the same time.