Tutorials

This tutorial will focus on interacting with usability test participants throughout a test session, from first contact to post-task activities. Two experienced test administrators will share their expertise and rules for successful interaction and, through examples, videotapes, and role-plays, illustrate their best practices for implementing the rules. Attendees will get to participate in role-plays and practice their skills.

PARTICIPANT KNOWLEDGE AND EXPERIENCE EXPECTED

We would like to have a mixture of experience:

• People who are new to test administration
• People with experience but who are self taught and want more thorough training
• Moderately experienced administrators who what to brush up on their skills

GOALS FOR THE SESSION:

Participants in this tutorial will:

• Learn a set of principals for interacting successfully with test participants and how to apply those principals
• Engage in role plays and get to describe their own experiences in interacting
• Take away a set of handouts on the principles of interacting, feel confident that they know how to interact with participants, and have ways to practice their technique

HOW THIS TUTORIAL WILL BE CONDUCTED

The presenters will describe their rules/principals for interacting with participants followed by the application of the rules before, during, and after the test participant works on the test tasks. Through demonstrations, videotapes, and role plays the presenters will illustrate the application of the rules and attendees will practice the techniques.

TUTORIAL SCHEDULE WITH TIME ALLOCATION

DETAILED DESCRIPTION OF TUTORIAL

The presenters have been administering test sessions and teaching students and relatively new usability specialists how to administer tests for many years. We know the literature and have contributed to it and have found that there is only sketchy information about interacting with participants. There are some general rules, but they are difficult to implement in specific situations and they don’t explain the typical difficulties that new testers have with such issues as being in control of the session, resisting becoming friends with participants, and dealing with their own rather than the participants’ anxiety over struggling with a usability problem. In fact, there is more literature about the unusual testing situations than there is about the typical ones.

We know that there are different styles of interacting, and that some testers are more effective than others, for example. But, we believe that most usability specialists would agree with the practices we plan to recommend in this tutorial. Of course, it is not our intention to present what we do as the only way to interact. In fact, we expect that some attendees will have their own ideas and that we will learn good techniques from them. Nevertheless, we do see a need for this tutorial, that is, for practitioners to lean basic rules for interacting with participants, why some practices are better than others, how to evaluate their own interaction, and how to avoid the bad habits we all fall into.

Our plan is to make the tutorial as interactive as possible and to vary the format. The variations will include:

• material presented by each of the moderators
• videotapes of testing situations with follow-up discussion
• role plays of testing situations with follow-up discussion

The first part of the session will consist of introductions and presenting the rules of interaction, followed by the issues involved in first contact with participants and the pre-test instructions and think aloud training. The next, and most important portion will focus on the interactions during the test session and some the tough issues such as keeping quiet, and giving assists.

Next, we will focus on the post-task activities such as how to get the most out of ratings and interview questions and we will discuss one of the issues that keeps test administrators from improving their technique, that is their reluctance to watch themselves.

SPEAKER BIOS

Joseph S. Dumas Ph.D.

Principal Usability Engineer

Oracle Corporation

Joe Dumas is a nationally recognized expert on usability testing and author ( with J. Redish) of A practical guide to usability testing . He currently conducts usability tests and manages usability method development for the Usability and User Interface Design group at Oracle Corporation. Prior to working at Oracle, he spent 18 years consulting on usability engineering methods with American Institutes for Research. His clients included the major high-tech industry leaders. He is also adjunct professor at Bentley College where he teaches a graduate course in usability evaluation. He has conducted hundreds of usability tests and thousands of test sessions over 25 years of being in the usability profession.

Joe has a PhD in Cognitive Psychology and is the author of two books and numerous articles for practitioners including a series of articles on usability methods for Common Ground , the original newsletter of UPA.

Beth Loring, CHFP

Director, Design & Usability Testing Center

Bentley College

Beth Loring, Director of the Bentley Design and Usability Testing Center, has more than 18 years of experience consulting on product design and usability. She has expertise in the areas of user needs assessment, graphical user interface design, usability evaluation methods, and ergonomics. She has designed and conducted usability tests of consumer electronics, web sites, delivery services and medical devices for diverse clients including Staples, Monster, Motorola, Samsung, and Baxter Healthcare. Beth has also been a visiting faculty member in Bentley's Information Design and Corporate Communications department since 2001.

Beth holds an M.S. in Engineering Design from Tufts University and is a Certified Human Factors Professional. She has published numerous papers on usability and is coauthor of a chapter in “Usability in Practice: How Companies Develop User-Friendly Products” (The Academic Press, 1994). She has received design awards from IDEA/Business Week, Comdex and ID Magazine . Beth is past chair of both the New England Chapter of the Human Factors and Ergonomics Society and the Greater Boston SIGCHI.