Register your product to gain access to bonus material or receive a coupon.
Critical Testing Processes: Plan, Prepare, Perform, Perfect
- By Rex Black
- Published Jul 29, 2003 by Addison-Wesley Professional.
Book
- Sorry, this book is no longer in print.
Description
- Copyright 2004
- Dimensions: 7" x 9-1/4"
- Pages: 608
- Edition: 1st
- Book
- ISBN-10: 0-201-74868-1
- ISBN-13: 978-0-201-74868-0
The advent of agile methodologies and test-driven development has brought software testing to the forefront of application development. Yet in today’s harried rush-to-market development environment, organizations must find a delicate balance between product release and product quality.
In Critical Testing Processes, the author distills knowledge gained from 20 years of testing experience into twelve critical processes. These include highly visible processes by which peers and management judge competence, and mission-critical processes in which performance affects the company’s profits and reputation.
After each process is introduced, the author demonstrates its use through an engaging case study. Instead of cumbersome regulations, this book provides checklists—lightweight, flexible tools for implementing process-oriented testing, gathering metrics, and making incremental process changes. By demonstrating critical processes in various organizational, operational, and technological contexts, this book shows readers how to:
Because testing is a collaborative process with the participation of staff throughout the organization, the author discusses interpersonal and cultural issues in depth. This book also devotes ample coverage to the often-overlooked areas of planning and perfecting tests. Whatever your role in testing—from test engineering to managing hundreds of test engineers—Critical Testing Processes will offer valuable insights into what you do, why it’s important, and how you can perform better.
0201748681B04222003
Sample Content
Downloadable Sample Chapter
Download the Sample Chapter related to this title.
Table of Contents
Foreword.
Acknowledgments.
Preface.
Introduction.
Book Scope.
Who Should Read This Book?
Demystifying Processes.
Types and Relationships of Critical Testing Processes.
Critical Testing Processes in Context.
Sumatra, a Hypothetical Project.
Terminology.
On Using This Book.
Recognize Good Processes.
Handle Challenges.
Implement Improvements.
To Your Testing Success!
I. PLAN.
1. Start with the Big Picture: Put the Test Role in the Broader Context.A Testing Process.
Understanding Operational and Organizational Context.
Introducing the Sumatra Project and the SpeedyWriter Test Team.
Testing within the System Development Lifecycle.
Organizing the Testers and the Testing.
Management Dimensions.
Beyond Testing Process Context.
2. Focus the Effort: Analyze Quality Risks.A Quality Risk Analysis Process.
Jamal Introduces the Team to Quality Risk Analysis.
Recognize a Good Quality Risk Analysis Process.
Involve the Right Participants.
Employ the Right Technique.
Separate the Vital Few from the Trivial Many.
Find Potential Bugs before Testing.
Start at the Right Time.
Adapt to Change.
Handle Challenges.
Getting People to Get Real about Risk.
Quality Risks from Various Sources.
Who Drives the Quality Risk Analysis Bus?
Eliminating the Appearance of Magic in the Quantification of Risk.
Dealing with Variation in Ratings.
Convincing Stakeholders to Invest Time and Accept Reality.
Implement Improvements.
3. Gaze into the Crystal Ball: Estimate the Work Ahead.An Estimation Process.
Jamal Predicts the Future.
Using Rules of Thumb for Rough Estimation.
Effort, Duration, and Dependencies.
Planning.
Staffing.
Test Development.
Test Environment Configuration.
Test Execution.
Moving on to Dollars and Cents.
4. It's Not What It Costs, It's What It Saves: Budget and Return on Investment.Analyzing Return on Investment for Testing.
Bugs That Get Fixed-or Prevented.
Bugs That Don't Get Fixed-but Are Known.
Tests That Mitigate Risks.
Guiding the Project.
A Final Note on Return on Investment Estimates.
Jamal Prepares the Bill-and the Justification.
Moving Forward, Then Looking Back.
5. From Estimate to Baseline: Obtain Commitment to Realistic, Actionable, Truthful Estimates.Selling the Estimate.
Jamal Makes His Case.
Recognize a Good Estimation Process.
Understand the Factors that Affect the Estimate.
Paint a Realistic Picture of the Project's Future.
Assign Responsibility for Action.
Predict with Honesty.
Estimate Quickly.
Handle Challenges.
What Kind of Investment Is Testing, Really?
It Might Look Like Magic-But It's Only an Estimate.
The Nine-Month Rule.
Be Careful with Nonstakeholders.
When Selling Fails.
Safety- or Mission-Critical Systems.
Implement Improvements.
6. Gain and Communicate Insights: Plan the Test Effort.A Test Planning Process.
Jamal Has a Plan.
Beyond the Process: Key Considerations for the Test Plan.
Selecting Strategies for Testing.
A Special Case: Regression Risk Management Strategies.
Understanding Test Environment Execution Resources.
Managing Test Systems and Environment Configurations.
Test Subproject Risks.
Getting Everyone On Board.
7. From Suggestion to Commitment: Assemble Stakeholder Support for Good Test Plans.Jamal Makes His Pitch.
Recognize a Good Test Planning Process.
Establish Clear Criteria for Phases.
Establish Consensus, Common Expectations, and Commitment.
Complete at a Reasonable Time.
Promote Reasonable Flexibility and Creativity.
Produce Appropriate Documentation.
Provide Opportunities to Catch Errors.
Exploit Synergies with the Project, Development, Build, and Integration Plans.
Handle Challenges.
Transcending the Template.
Planning for Outsourcing.
When Key Players Don't Support the Test Plan.
Obeying the Law and Pleasing Regulators.
Implement Improvements.
II. PREPARE.
8. Bring on the Great Testers: The How and the Who of Hiring.A Team Building Process.
Variations on the Team Building Process.
The Sumatra Test Team Grows.
Identifying and Managing the Critical Test Team Skills.
Regarding Test Technicians.
Test Team Staffing Variations: Temporary Assignment, Rotation, Farm Team, and Backwater.
Education, Training, Certification, and the Professionalization of Testing.
Attitude Matters Too.
Beyond the Obvious Interview Questions.
From Adding People to Growing Skills.
9. Grow Excellent Test Teams: Skills, Attitudes, and Career Paths.Jamal and Lin-Tsu Discuss Skills Growth.
Using Skills Assessment as a Career Growth and Team Management Tool.
Recognize a Good Team Building Process.
Operate on a Win-Win Philosophy.
Accurately Define the Job.
Provide the Employee with a Long-Term Career Path.
Give Employees Appropriate and Fair Salaries.
Retain Employees with Sufficient Education, Experience, and Skill.
Allow Appropriate Specialization.
Handle Challenges.
What Do Testers Need to Know-and What Can You Get?
White Elephants.
Using Contractors and Consultants.
Implement Improvements.
10. Archimedes' Bathtub: Design and Implement Test Systems.A Test System Design and Implementation Process.
Emma Creates Stress on the Job.
Three Crucial Considerations.
Test Oracles.
Appropriate Documentation.
Combinatorial Explosions.
Shifting to the Management Picture.
11. Fill the Bathtub: Test System Coverage and Quality.A Brief Survey of Test Coverage Analysis Techniques.
Jamal Assesses Coverage.
Recognize a Good Test System Design and Implementation Process.
Provides Capable Tools for Assessment of Quality.
Efficiently Implements, Uses, Reuses, and Maintains the Test System.
Selects the Right Techniques.
Prevents Bugs.
Handle Challenges.
How Many Conditions in a Test Case?
Retesting Third-Party Components.
Vaguely Defined Requirements, Usage Profiles, and Environments.
Balancing Coverage, Schedule, and Budget.
Implement Improvements.
III. PERFORM.
12. An Essential Handoff: Manage Test Releases.A Test Release Process.
The Big Build Hits the Test Lab.
Recognize a Good Test Release Process.
Provide Installable, Improved, and Stable Test Releases.
Produce Predictable, Timely, and Regular Test Releases.
Use Defined, Customer-Like Install and Uninstall Processes.
Give the Test Release a Name That It Knows.
Build from Carefully Selected, Coordinated, and Documented Content.
Handle Challenges.
Who Owns the Test Release Processes?
Complexity of the System Under Test-and Beyond.
Incomplete and Unrealistic System and Acceptance Test Releases.
Implement Improvements.
13. Assess Quality: Execute Test Cases.A Test Execution Process.
Test Tracking Worksheets.
The Attack on the Big Build.
Recognize a Good Test Execution Process.
Find the Scary Stuff First.
Produce, Gather, and Disseminate Valuable Information.
Correctly Interpret Test Results.
Exhibit the Proper Attitudes.
Proceed Efficiently.
Handle Challenges.
Dealing with Extended Shifts and Outsourced Testing.
Accommodating Holidays, Cultures, and Vacations.
Capturing the History on the Ground.
Implement Improvements.
IV. PERFECT.
14. Where Quality Falls Short: Report Bugs.A Bug Reporting Process.
A Big Bug in the Big Build.
Beyond the Failure Description.
Recognize a Good Bug Reporting Process.
Effectively Communicate Useful Information.
Describe One Symptom per Report and File One Report per Symptom.
Draw a Clear Boundary Between Testing and Debugging.
Handle Challenges.
Separating Bugs, Features, and "So What".
Handling Bugs That Got Fixed by Accident and Irreproducible Symptoms.
Keeping Noise Out of the Signal.
Building Trust and Credibility with Programmers.
Selecting the Right Bug Tracking Tool.
Implement Improvements.
15. Illuminate the Fourth Element: Report the Test Results.A Test Results Reporting Process.
Jamal Defines a Dashboard-and Reports on the Big Build's Test Results.
Recognize a Good Test Results Reporting Process.
Deliver Useful, Credible, and Timely Information.
Relate Test Status to Project Progress-and Influence Future Progress.
Show Trends and Destinations in Quality over Time.
Use Appropriate Mechanisms and Documents.
Tune the Communication to Each Listener.
Handle Challenges.
Handling the Presentation.
The Bearer of Bad News.
An Ever-Changing Complex Story.
Maintaining a Passion for Quality While Giving a Dispassionate Presentation.
Implement Improvements.
16. Leverage Opportunities to Learn: Manage Change and Its Effects on Testing.A Change Management Process.
Jamal Makes a Case-and Accommodates.
An Interconnected Process.
Recognize a Good Change Management Process.
Select the Right Changes in the Right Order.
Balance Considerations of Features, Schedule, Budget, and Quality.
Handle Challenges.
The Complex Nature of Change Effects.
The Ripple Effects of Change.
Becoming a Roadblock-or Becoming Perceived as a Roadblock.
Implement Improvements.
17. Return to the Big Picture: Perfect the Testing Process.Recognize a Good Test Process.
Provide Valuable, Economical Services.
Pervade the Project.
Use a Mature Process.
Apply the Appropriate Level of Formality and Standardization.
Employ Appropriate Strategies.
Successfully Introduce Tools.
Enjoy Continuous Learning and Inspiration.
Handle Challenges.
How Can We Measure What We Can't Define?
Immature Development or Maintenance Processes.
Unrealistic Expectations.
Relationships.
"Addicted to Test".
Incremental Process Improvement.
Fine-Tune the Big Processes.
Identify and Remove the Big Process Obstacles.
Implement Improvements.
Jamal Brown Looks Back-and Looks Forward.
Conclusion.
Glossary.Bibliography.
Index.
Preface
I’ve spent most of my twenty years in the software and hardware business inthe arena of testing. For the first few years as a test practitioner, I struggled tokeep my head above water. Ultimately, I mastered some basic tools and techniques.
As I learned more about testing, I started to notice certain common themes.Some of these themes had to do with events—good and bad—that happenedover and over again on software, hardware, and system projects. For some of theseevents, I found that some teams could create order in their projects. These teamshandled these common events better than the teams that bounced from one crisis tothe next, reacting constantly, immersed in chaos. The successful teams had goodprocesses.
Some of these successful project teams implemented writtenprocesses, while others accumulated “institutional knowledge” in theirwise—and sometimes prematurely gray—heads. While I have nothing againsta shared company culture, it’s hard to pass along the processes you’velearned unless you write them down-whether formally or informally, as checklists.This book takes the informal road. I describe twelve specific test processes, usingchecklists.Each process is critical to test team success.
I describe theseprocesses in chronological order. First we plan the test activities. Next we prepareto test. After that, we perform the tests. Finally, we perfect the system under testand the testing activities themselves.
Many other books have covered thetopics of preparing and performing tests in great detail. My experience is that, astesters, we generally do a good job in these areas. So, instead of rehashing what wealready know, I focus on opportunities for improvement. I devote eleven of theseventeen chapters to the topics of planning and perfecting. By far, these are theareas where we as testers have the most difficulty. This is especially true forcomplex and critical projects.
Where will this book take you? During theearly colonization of the American continent in the 1540s, Francisco Vasquez deCoronado searched the deserts of present-day Arizona and New Mexico for the SevenCities of Cbola, including El Dorado, a city whose streets were supposedly pavedwith gold. Juan Ponce De Leon searched for a Fountain of Youth. In 1911, one of thefirst management consultants, Frederick Winslow Taylor, wrote a book called ThePrinciples of Scientific Management. Taylor espoused the idea of the one bestway—the perfect process—for each activity on an assembly line or in anyother industrial enterprise. But none of these three men found streets of gold, lifewithout death, or perfect processes.
This book isn’t about Quixoticquests. There are no streets of gold that will make us effortlessly rich. Wecan’t side-step our human limitations. I don’t have infallibleprocesses. As Frederick Brooks wrote in the Mythical Man-Month, SecondEdition, we don’t have any silver bullets to kill our system projectmonsters, including the ones that live in quality and testing. That said, I havefound many ways for testers to deliver valuable information and services to theproject team, and each of these ways has its strong points and its weaknesses.
The processes in this book might differ from what you’re doing now. In somecases you’ll decide, based on the success of your current processes, thatyou’re doing a fine job already. In some cases, though, you may want toimplement improvements. I’ll discuss specific ways to do that, but two themesapply to process change throughout the book.
First, only change what’sbroken when changing it will help. Process change for its own sake, or processchange to perfect an already-good process, often doesn’t help the test team orthe organization. Indeed, such efforts can prove a dangerous distraction fromwhat’s truly important.
Second, change is often easiest when done insteps wherever possible. Change should be made as painless as possible. All theprocesses in the book were developed through incremental change as I realized that abetter way of doing things would significantly increase the value my team could add,and fine-tuned my processes to achieve that.
The processes in this bookaren’t pie-in-the-sky theory, but rather grew out of my experiences on theground as a practicing tester, test lead, and test manager. Your experiences andyour challenges will differ from mine, so be sure to adapt my processes—orcompletely reinvent your own—rather than trying to put a saddle on a cow.Following good processes can liberate you from the rote aspects of certain tasks,allowing you to focus on the fun, the fascinating, and the creative. When theprocesses you’ve adopted no longer solve the critical problems, when they needto evolve as your situations change, when they get in the way, then it’s timeto rethink how you do what you do. The processes I discuss here are lightweightchecklists (things I want to remember to do), not bureaucratic regulations (things Ihave to do because someone told me to).
I hope that this book will start youthinking about the following questions: How do we do our testing jobs every day andon every test project as best as we possibly can? How do we de-institutionalize ourknowledge of how we do what we do? Even though we have varied experiences, is therea commonality of practices that we can share for critical testing processes thatdetermine our success? This book will give you a compendium of proven testingprocesses to help jump-start the most critical testing process of all, the thinkingprocess.
0201748681P04072003
Index
Download the Index
file related to this title.