This chapter is from the book
This chapter is from the book
This chapter is from the book
Examples of Testing Practices
I have already described system testing, where professional testers take the whole application and methodically go through the use cases looking for unexpected behavior. This sort of testing can be automated to some extent with iOS apps, using the UI Automation instrument that’s part of Apple’s Instruments profiling tool.
System tests do not always need to be generic attempts to find any bug that exists in an application; sometimes the testers will have some specific goal in mind. Penetration testers are looking for security problems by feeding the application with malformed input, performing steps out of sequence, or otherwise frustrating the application’s expectation of its environment. Usability testers watch users interacting with the application, taking note of anything that the users get wrong, spend a long time over, or are confused by. A particular technique in usability testing is A/B Testing: Different users are given different versions of the application and the usages compared statistically. Google is famous for using this practice in its software, even testing the effects of different shades of color in their interfaces. Notice that usability testing does not need to be performed on the complete application: A mock-up in Interface Builder, Keynote, or even on paper can be used to gauge user reaction to an app’s interface. The lo-fi version of the interface might not expose subtleties related to interacting with a real iPhone, but they’re definitely much cheaper ways to get early results.
Developers, particularly on larger teams, submit their source code for review by peers before it gets integrated into the product they’re working on. This is a form of white-box testing; the other developers can see how the code works, so they can investigate how it responds to certain conditions and whether all important eventualities are taken into account. Code reviews do not always turn up logic bugs; I’ve found that reviews I have taken part in usually discover problems adhering to coding style guidelines or other issues that can be fixed without changing the code’s behavior. When reviewers are given specific things to look for (for example, a checklist of five or six common errors—retain count problems often feature in checklists for Mac and iOS code) they are more likely to find bugs in these areas, though they may not find any problems unrelated to those you asked for.