- Consumer Purchase the Hard Way
- Modern Savages
- What's the Problem with Software?
- Review Without Peer
- Bad Alternatives to Peer Review
- Preserving Our Peer Processes
What's the Problem with Software?
Software sits at the top of an information pyramid. When a software problem or opportunity is first conceived—for example, the need for a spreadsheet tool—there is no software. Instead, unlabeled in our subjective experience lie all the matters we personally apprehend about the world, like the desire for a roof over our heads. Unlabeled in another bucket, which we'll call the objective world, lie all the matters that constrain us without our permission, like the need to cross roads carefully. Through research, whether academic, commercial, marketing, personal, or entrepreneurial, some of these matters are brought into the foreground. From such matters come ideas, from ideas come plans, and from plans come lines of code that make up software products. Before too long you have a VisiCalc spreadsheet program, then a Lotus 1-2-3 spreadsheet program, then Microsoft Excel, then OpenOffice.org. The software, built as a solution to the problem of which we were originally mindful, goes to market; those of us with enough interest acquire it for use. But somewhere in this process of distilling vague ideas into concrete software, things are less than perfect.
The software development process is engineering of sorts, and the way wrong turns are identified in software engineering is to burden them with the label defect. Defects are software features that aren't wanted, but that crept in anyway. It's a bit narrow to think of software solely in terms of "defect density," but that's the stick that gets waved about.
In the information pyramid, it's very interesting to note that defects can creep in at every stage of the software deployment process—from blue-sky research to consumer adoption. The two that we're most anxious about these days are defects in execution, commonly called bugs, and defects in intent, which are hostile features. Both are "put into" software—to use the engineering parlance—by technologists or programmers. Both are put in during the software build part of the process.
The best defense against a high defect density is review. Two heads are better than one, and review provides fresh perspective that can locate flaws that are otherwise ignored. The best kind of review is independent peer review, because then there's no collusion or ignorance to hide behind. Independent peer review is a widespread process tool. Lawmakers use it. The medical profession uses it. Science uses it. Accountancy uses it. Independent peer review is a standard instrument used to test the probity and transparency of any process that depends on human factors.
It's interesting to note that independent review already occurs at many points in the software information pyramid. Academic review of papers, black-box testing of software, and market acceptance of final product are all examples of peer review with some degree of independence. There is a very weak link in the chain of review checks and balances, however. While software is being built—after the academia is over and before market acceptance is tested—frequently we have weaker review. This is a hole in the process through which defects can climb in. This is also a significant reason why software has such a bad reputation.