Surviving the Top Ten Challenges of Software Testing: Challenge #5: Making Time for Testing
Overview
Testing is a paradox. If we could just test the parts of the system that we suspect of having defects, we could drastically reduce the amount of time spent on testing; but defects are usually found in the places we least suspect—so we dare not assume that any area of the system is defect-free. As they say, a good tester always looks both ways before crossing a one-way street.
If all areas of the system are suspect for defects, it follows that all areas of the system should be tested. But, to what extent should the system be tested? When you consider all of the possible input combinations, there is an astronomical number of test cases required. This leads us to a basic realization about testing: When brute force is used, there is always more work than we could ever possibly complete. No matter how many tests you design and perform, there always seems to be one more. The exception is software that is very small in scope and complexity, which is a rarity in the real world.
Many times, the testing schedule is a “best guess” added to the end of a project to meet an arbitrary deadline. Instead of estimating the testing schedule based on the number of test cases to be performed, the size of the system, or the number of requirements to be tested, the estimate is based on some fuzzy idea of what seems reasonable. Testers are then put under pressure to meet a testing schedule that is severely underestimated.
There are also variations on this theme, such as the situation when there is no major project, only continuous maintenance, as in legacy systems. In that case, independent testers are expected to keep up with the work of developers. Many times, this work load can be overwhelming and the pressure to complete the test can be relentless.
An interesting question is, “What would happen if management gave testers all the time they needed for testing?” Probably, there would still not be enough time for what many people consider complete testing. Somewhere between the agony of rushed deadlines and the luxury of all the time in the world has got to be a reasonable approach to testing. That’s what the rest of this chapter is all about—making the most of the time you have available.