- Introduction to Your Toolkit
- Testing Facilities
- Recording of Testing Sessions
- Modeling Tools and Software
- Data Gathering and Testing Techniques
- The Special Needs of International Testing
- Recruiting Interview and Testing Participants
This chapter is from the book
This chapter is from the book
This chapter is from the book
Recruiting Interview and Testing Participants
Usability tests typically require fewer participants than marketing research studies because the findings in usability tests are usually qualitative, rather than statistically descriptive. In usability testing, you are not trying to generalize your results and estimate the numbers or percentage of people who feel or would react to a product in a certain way. You are exploring. You are trying to determine whether there are usability issues, what they are if they do exist, and how you might solve them. This means you are trying to delve into the psychology of your users. And this requires that the participants you test are representative of the target population of actual users. You are going to need to find representative users for data gathering and usability testing.
In-house users, while easy to find, aren't usually acceptable participants because they probably care more about the company than the real users do. They see the application as being worthy of additional effort and might exaggerate its value, or they might not flag aspects of the design that make it impractical. They are also familiar with the company's in-house language, concepts, attitude, and mindset, and they might even have different aesthetic values and perceptions than typical end users.
In one case it is fine to perform tests with in-house users: If you are actually building an application for the internal staff members, it is appropriate to sample them. This is usually a very easy and informal process; the staff members just need to be screened and scheduled.
Lots of market research and usability testing companies have staffs of screeners—clerical-level staff who call lists of potential participants and follow the questions in a special questionnaire (also called a screener, as noted earlier in the chapter). The staff members use the questionnaire to select participants who fit the criteria for the study. Participants are typically offered a fee of $100 to $200 each, depending on how stringent the required match criteria are. Some of these facilities have databases of potential participants. This can be convenient, but the lists may be overused. (Some people seem to be making a part-time job out of participating in studies!) You may want a fresher list. To accomplish this, you may need to ask the recruiting firm about the people in their databases. You can shop around for databases and recruiters, and you can specify that the participants must not have been in any studies during the last 12 months. This may make your recruiting more expensive because it may be harder to find participants. If you need general participants, for example, people between the ages of 20 and 60 who purchase goods from the Web at least once every 3 months, it may be relatively easy to find "fresh" participants. If you need people who work in a copy center who have never used a particular type of software, you will pay more for this type of recruiting. It is good to have relationships already set up with companies that can help you recruit participants.
If your user group is current customers, it may be possible to develop a list of customers and have the staff screeners work from that list. This may be easier and more cost-effective than using a recruiting firm. In some cases, you can have internal staff work temporarily as screeners. This costs very little unless you need to hire in-house staff to work as screeners full time. Using in-house screeners saves money over hiring a screener consulting firm, but the in-house screeners will need to be trained. Usability consultants are already trained and just charge you per project. But having a smooth machine for obtaining study participants helps keep usability work progressing—problems with obtaining participants is the single most common source for the delay of usability projects.
A whole series of deliverable documents result from proper usability work. It is true that some people approach usability without much of a concept of deliverables. They think they can just study the user and good things will happen. That may be true—good things may happen. But to make usability work efficient and repeatable requires an organized set of deliverable documents. The deliverables give a clear focus and a set of milestones for usability work. As an example, Table 8-1 lists the major deliverables in The Schaffer Method.
Table 8-1. The Major Deliverables in The Schaffer Method
|
Schaffer Method Deliverables |
|
|---|---|
|
Expert Review Report Usability Test Report Kickoff Presentation Data Gathering Plan Insight of the Data Gathering Contract for Design User Interface Structure Standards Identification Custom Standard Module Definition Module Requirements Screen Flow, Functions, and Fields |
Wireframes Graphic Treatment Simulation Test Results Functional Specification Developer Briefing Design Modification Log Final Usability Test Plan for Process Improvement Post Release Evaluation Localization Assessment Interface Translation Specification |
It takes time to create good deliverables, but they offer several benefits.
-
They document that steps in the methodology are actually completed.
-
They allow work to be communicated to others, for instance, key stakeholders and development staff.
-
They allow work and processes to be repeated.
Most deliverables require several smaller deliverables to create the end product. So, in the end, there are hundreds of deliverables. Imagine that you needed to create these deliverables from scratch for each project, figuring out the appropriate document structure and inventing the style of presentation. The level of investment for this would make usability engineering programs prohibitive in cost and time. If each of the 23 deliverables listed in Table 8-1 took just 1/2 a day to create structurally, then you would add 111/2 days to the project.
If usability is to be routine, standard reusable deliverables are indispensable. They help organize the project and save valuable time. Standard deliverables also make it is easier for managers to check a project's progress because they know the full set of deliverables to expect. Finally, using standard deliverables also makes it easier to get oriented and to review an unfamiliar project.
The value of the tools, templates, and facilities outlined in this chapter is that they save you valuable time. However, it remains critical to pick the items most appropriate for your efforts. It is not sensible to invest in something just because it is a new technology. Refer back to your strategy often, and remember to let your methodology determine your toolkit. The next chapter provides information on another valuable time-saver—the implementation of interface design standards.