Introducing Software Testing

Description

Introducing Software Testing introduces practical ideas for a software tester to jump-start the testing effort. Strategies presented tackle the common obstacles of testing in order to meet time critical deadlines.  The examples included walk the tester through the concepts presented, including how to design tests for products that have insufficient requirements. Documentation is essential to the success of testing software and recording accurate results. Risk analysis is covered to help the tester identify the most relevant tests to address the most important features.  

Extras

Related Article

Software QA 101: The Basics of Testing

Sample Content

Table of Contents

(All chapters start with an Introduction and end with a Summary)
1 Tackling The Testing Maze
2 Test Outlines
3 From Test Outline To Test Cases
4 Using Tables And Spreadsheets
5 Other Types Of Tables
6 Testing Object-Oriented Software
7 Testing Web Applications
8 Reducing The Number Of Test Case
9 Creating Quality Software
10 Applying Software Standards To Test Documentation
Appendices
References
Index

 

Preface

The chaotic testing environment

A project is in panic mode and the deadline is rapidly approaching. Management starts to think about the need to test this product, having already missed some prime opportunities for improving software quality. One unfortunate programmer is assigned the task of software testing, which is often viewed as being transferred to purgatory. Needless to say, this poor hapless soul is given no guidance, and nobody in the organization is capable of providing any help. Despite the poor condition of requirements and other product documentation, the product is being built and it will be shipped. The task given to the tester is to minimize the surprises that could manifest themselves after the product is installed at customer sites. Under extreme pressure, this untrained tester is very inefficient and is at a loss how to begin. A clueless manager may even purchase testing tools, despite there being no useful tests to automate. This is the scenario that gives software testing a bad name.

Software testing is a specialized discipline requiring unique skills. Software testing is not intuitive; one must learn how to do it. Naïve managers erroneously think that any programmer can test software — if you can program, then you can test. This is the motivation behind this book: to provide a step-by-step approach for getting started with the testing effort.

Many fine books on software testing are available today. Those that do address test case design describe proven methods such as boundary value analysis, equivalence class partitioning, decision tables, syntax testing, cause–effect diagrams, data-flow methods, and other such concepts. Some novice testers wonder how to weed though poor specifications, before even being able to apply these methods. Many texts state that good requirements are necessary for the test effort — assuming that requirements exist — yet I have not seen any that explain the transition from requirements to test cases. In the chaotic software development environment, adequate requirements are rarely provided, and if they are, their completeness and correctness are questionable. In a situation when no one has analyzed the requirements adequately, the burden falls on the tester to pursue requirements issues prior to defining any tests. It is often impossible to perform thorough testing, given the tight schedules and limited resources. It is possible, however, to make intelligent choices and maximize the effectiveness of the testing effort.

The goal is to learn how best to approach the testing tasks and eventually produce a workable test process for future projects.

The author’s philosophy

To introduce the ideas on how to begin testing, I will work through several detailed examples, each containing defective "requirements". By definition, good requirements are testable and unambiguous. The fact that the sample requirements are deficient does not prevent useful test activities from occurring. I use the word "requirements" loosely and equate it with some sort of product description. While the sample test scenarios would not be permissible in a mature software organization, the work described will help the lone tester jumpstart the testing process under duress. The goal is to show that some product information, however deficient, can be used to start the testing effort.

I do not advocate working from poor requirements. Properly analyzing requirements corrects many deficiencies. Reviews and inspections have been proven to provide the most cost effective method for finding problems early in the development cycle. Many times, I have had to bite my tongue to avoid blurting out to project managers, "The requirements are absolute garbage and there’s no way that we can begin a productive testing effort until you clean up your act." Actually, this phrase would contain unprintable language and be uttered under one’s breath. We have undoubtedly all shared this fantasy, and the ugly truth is that despite this valid complaint, the product delivery deadline is fast approaching.

Although I do not advocate cutting corners, there are some shortcuts that will help document the testing activities. A crude list of tests is better than no list. The minimum you will have is a documented trail, though rudimentary, that records your testing effort should you need to prove or demonstrate what you did.

Subsequent testing efforts will improve on this initial work, producing test documents and developing a test process that is more in line with accepted practices. Incremental changes lead to successful process improvements.

Just knowing how to get started with testing is a feat in itself. The tester must understand how to transform product information into test cases; this is the book’s chief goal. Many existing books do an outstanding job of explaining software testing concepts and methods. Rather than reiterate what others have written, I make many references to their work. This book is a primer on getting started. It supplements currently available literature on software testing by providing an introduction to known software testing techniques.

Intended audience

This book is aimed at several types of readers:

• persons new to software testing who have no guidance or training;
• managers or mentors, who may themselves be experienced testers, seeking ideas on how to provide guidance to novice testers;
• experienced programmers who have been assigned testing tasks;
• knowledgeable testers looking for new ideas.

While readers are not assumed to be knowledgeable about software testing concepts, they should be computer literate and able to use a word processor and spreadsheet.

Job descriptions

The general job description terms used throughout the book are as follows:

• Tester: The person who defines and executes tests.
• Developer: The person who produces the application, including the design, source code, and final product integration.
• Project manager: The person with authority regarding schedules and staffing.
• Project authority: The domain expert with authority to define and clarify the requirements.

I refer to these descriptive titles without implying an organization structure or employee reporting chain. Project staffing decisions and job responsibilities vary across organizations.

Depending on how the project is staffed, the tester could be either in the same or in a separate group from the developer. Other projects could require that the same person performs both development and testing tasks, thereby changing mindset in mid-project. Ideally, a trained software test engineer performs the testing activities. However, some projects simply assign the testing role to whichever person is available.

The project authority can be the marketing manager, company executive, or customer support liaison, provided that this person has full authority to define the project contents. This role is necessary to prevent further chaos. Someone must be in charge of deciding which features to incorporate into a product; lack of such control is a well-known cause of problems when trying to get bad requirement definitions sorted out.

Your organization may use different job titles than those listed above. The key point is to assign people to perform the necessary tasks — each of which requires specialized skills.

Mature and immature software development environments

The examples cited in this book, with their incomplete product information, are what one could expect to find in an immature software organization. I will refrain from critiquing the work environment and from preaching about software process improvement. The reality is that many companies operate under less than ideal conditions. Despite the lack of suitable software processes, products are still being developed and shipped to customers. Testing, however minimal, can still be done. With poor requirements, the tester spends more time identifying product definition deficiencies rather than proceeding with testing-related tasks.

A mature software organization displays the characteristics listed below. An immature organization often does not understand how the following points can improve product quality:

• provide useful requirements and product descriptions;
• conduct reviews and inspections;
• have signoffs or checkpoints before proceeding to the next step;
• mentor and train personnel;
• schedule adequate time and resources for testing;
• overlap testing and development activities;
• provide defined software development and software testing processes;
• enforce configuration management.

Testing is a responsibility shared with the rest of the development team. The old view of testing as an afterthought — design, code, and then you test — has never produced good testing results. The adversarial and destructive "developer vs tester" mentality has often resulted from the developers’ ignorance about software testing activities — more proof that testing is a unique discipline. It is often the case that a tester often knows more about programming than a developer knows about software testing. A collaborative approach between testers and developers fosters goodwill and good communication. By working closely with the testers, many developers learn more about software testing, even if all the developers see is how their knowledge about the product filters into the test documentation. Effective software testing requires co-operation among all the members of a project.

Book overview

Chapter 1 deals with the unfortunate "you’re new to testing, have no idea where to start, and the product ships Friday" nightmare. Hoping that your next project gives you more time to carry out testing activities, Chapter 2 illustrates the use of outlines, which is also a useful technique for analyzing requirements if no one else has done this task. Chapter 3 transforms the outline contents into test cases. Tables and spreadsheets are an integral tool used by software test engineers, and Chapter 4 shows several table formats and shortcuts for documenting test cases. Chapter 5 shows additional usages of tables. Applications built using object-oriented methods can use many of the same test design techniques outlined in the previous chapters. However, Chapter 6 describes some issues particular to testing object-oriented systems. Chapter 7 lists the challenges faced when testing web applications, although many of the strategies presented apply equally to client-server environments.

No uniform software testing method exists. Each example uses a different approach for producing tests. You may wonder why one method was used in one example instead of another. The answer is simple: I selected a method based on my experience. You may very well try a different approach that will be just as successful in your testing effort. Although the examples cover different types of applications, the core software testing themes apply equally to all examples, and some concepts are reiterated among all chapters. I recommend that you read through each scenario and not dismiss the subject simply because the example does not reflect your type of application.

By following the ideas and methods presented, you will have defined and documented many test cases. Chapter 8 will help you identify the most pertinent tests and thus reduce the necessary number of test cases to execute. Producing a set of test cases to execute is only part of the overall software testing picture. Chapter 9 lists other testing and quality related tasks that are necessary for producing quality software.

If this is the organization’s first venture into methodical software testing, you will have established a good baseline. Although the work produced will be a vast improvement over prior chaotic efforts, it will fall short of satisfying, and conforming to, industry standards. Chapter 10 briefly describes some of the more common software engineering standards and how each affects the test case examples. Consider this a launching point for improving the testing effort.

Updates

Submit Errata