Tips for Usability Professionals in a Down Economy
Journal of Usability Studies, Volume 4, Issue 2, February 2009, pp. 60-69
Article Contents
Tip #1: Be More Efficient with Your Usability Tests
We're always trying to improve the efficiency of our work, but that's even more important when resources are tight. Many of us work in companies where staff has been reduced and budgets cut, but we're still asked to keep up with the same demand that we had before the cuts. Traditional usability testing, whether in a lab or remotely over the phone, tends to be pretty time-consuming. Very useful and important, but not very efficient. It tends to be time-consuming both for the usability people running the test sessions and for any project team members observing them. The following are several ways that you might be able to improve the efficiency of this process:
- Staggered test sessions. One technique that can help improve efficiency for the observers, if you can do it, is to schedule overlapping and staggered participant sessions. For example, assume that the start-up time for a participant session, including the briefing, informed consent, session instructions, etc., is about ½ hour. And the main part of the session (interacting with a prototype, doing tasks, giving feedback) takes another ½ hour. So the idea would be to start one session say at 9:00 and then another, parallel, session at 9:30. The observers would then "tune in" starting at 9:30 to watch the main part of the Participant 1 session, switching to the Participant 2 session at 10:00. This could continue, with the observers experiencing very little downtime during the day. Of course this requires more than one usability person to facilitate the parallel sessions, and it requires the technology or facilities to support two simultaneous sessions. This isn't really any more efficient for the usability people, but it does allow you to get the testing done in about half the time, and the entire process appears much faster and more efficient to observers and other business partners.
- More tasks, not more participants. I've lost track of the number of times I've been asked how many participants you really need for a usability test. We find that our business partners are usually the ones pushing for more participants in a test. ("What, you want to test with only 6 or 8 participants? How can that possibly be valid!?") Tight times are when you especially want to push back on the desire to test with 15 or 20 participants when you're confident that 6 or 8 is sufficient. One piece of evidence you can point to is the analysis by Lindgaard and Chattratichart (2007) of the data from Comparative Usability Evaluation-4 (CUE-4, Molich & Dumas, 2008). They looked at the results from nine teams that conducted independent usability tests of the same Website. They identified the superset of usability issues from all of the tests. Then they looked at the correlations between the percentage of that full set of issues for each test and the number of participants in the test. There was no correlation. But there was a significant (positive) correlation with the number of tasks used in the test. More tasks uncovered more issues, while more participants didn't. One technique you can use to get greater task coverage is to identify a core set of tasks that every participant will be asked to do and another set of tasks that you would select from for each participant. Selection could be done randomly, by some rotation, or by the participants choosing the ones most relevant for them.
- Put the observers to work. A final tip related to traditional usability testing is to make the observers do some work. A technique that's becoming popular is to ask the observers to write down usability issues on sticky notes as they observe them. Then between sessions they post them on a white board, grouping similar issues if they have time. At the end of the sessions, the usability person could facilitate a session where the issues are further grouped, refined, and even prioritized. These can then be easily incorporated into a final report or presentation that includes additional information from the sessions (e.g., task completion rates, subjective ratings, other information that may not have been obvious to the observers). Obviously this technique works best when all the observers are in the same room. (We often provide a room for the observers to gather in even when the sessions are remote.)