| |
|
|
|
|
|
Workshop 3: Beyond the Usability Test: Advanced Usability Issues and Methods
Stephanie Rosenbaum, Judith Ramey
Tuesday, July 9th, 8:30 AM - 5:00 PM
Contact: stephanie@teced.com
Audience: Advanced
Abstract:
This workshop focuses on three major topics: methods for addressing difficult issues, variability in the definition and application of methods, and combining methods into usability programs. Its goal is to increase practitioners' abilities to analyze problem areas in products and systems, as well as choose the most effective usability methods.
Product Category:
No specific system, product, or project orientation
Topic Category:
Issues and strategies for experienced usability professionals
Topic Focus: Combining methods, Professional development, Strategic usability
Duration:
Full day
Description of Workshop:
The practice of usability research has matured since people from many disciplines (human factors, cognitive psychology, technical communication, anthropology, and more) began to work as usability practitioners. We now recognize that user experience issues are much more complex than was believed in the first years of usability practice. It is important for experienced usability practitioners to address these challenges if we wish product and system design over the next decades to be guided by data about users and their goals, tasks, and environments.
Improving the range and diversity of usability methods will also enlarge the contribution we can make to the development process. The more of the designers' and developers' questions we can answer with user data, the more it is likely that usability practitioners will become key members of product development teams.
Therefore, the primary goal of this workshop is to add richness and depth to usability practitioners' expertise by sharing with other workshop participants their experiences with various methods beyond the usual "thinking aloud" usability test, including the strengths and weaknesses of these methods. The workshop is intended to increase the participants' ability to analyze problem areas in products and systems, as well as to help the participants choose the most effective usability methods or combinations of methods.
The workshop will focus on three main topics:
- Methods for addressing difficult issues
Participants will identify difficult or challenging issues on which they conducted user research. They will describe-and the group will discuss-the methods they used to address the issue, how well these methods worked (what characteristics of the methods worked well and what did not work well), and what other issues emerged as a result of exploring this one.
- Variability in the definition and application of methods
The group will try to learn how much usability methods vary among and across members of the usability community. Participants will describe-and the group will discuss-how they measure inter-rater reliability, how they categorize research methods, and how they classify research subjects.
- Combining methods into usability programs
Participants will describe usability programs they have developed. The group will discuss how to balance the strengths and weaknesses of methods when building usability programs, how to select and combine methods to increase the reliability of an overall program, and how to ensure better coverage (identify more types of problems, miss fewer issues) when building a longitudinal usability program.
Schedule of Events (assuming 7 actual hours of workshop, can be expanded or tightened as needed):
Presentation and discussion of position papers: 1-1/2 hours
Categorizing questions and open issues from position papers: 1 hour
Discussion of methods for addressing difficult issues: 1 hour
Discussion of variability in definition and application of methods: 1 hour
Discussion of combining methods into usability programs: 1-1/4 hours
Summary of workshop conclusions and remaining open issues: ¾ hour
Wrap-up and agreement on action items: poster, more: ½ hour
Note: This workshop is revised from one the organizers presented at the IHM-HCI 2000 conference in Lille, France. This UPA workshop will better reflect the organizers' original intentions to share and compare the methods used by experienced practitioners. Although the IHM-HCI 2000 workshop was productive, all the participants did not have the level of industry practitioner experience that many senior UPA members have; we hope to accomplish yet more in this UPA workshop.
Participant Selection Criteria:
The facilitators will select the workshop participants based on their position papers, choosing participants who have experience in a variety of methods and have learned about the strengths and weaknesses of different methods. Our preferred workshop size is 15 participants; we'll accept up to 20 if we receive that many position papers indicating the participants will contribute to the workshop interactions.
Position Paper Required for Acceptance
Workshop registration is closed. If you are interested in participating please contact the workshop coordinator immediately. (Previous registration information: You must submit a position paper for this workshop before your workshop registration will be accepted and processed. Send your position paper to stephanie@teced.com. Please include the following information in your position paper:)
Pre-Workshop Participant Activities:
Prospective participants will write a position paper (2 - 3 pages) describing their experience or interest in advanced usability issues and methods, including a summary of a relevant usability project they have performed.
Pre-Workshop Facilitator Activities:
The facilitators will analyze the position papers and develop a detailed list of issues and questions to address during the workshop.
Post-Workshop Dissemination:
If the workshop is accepted, the organizers will submit a proposal for a UPA 2002 poster summarizing the outcome of the workshop. As described in the submission guidelines, we will also produce a workshop report in the form of an article suitable for publication in the UPA's new magazine, User Experience.
BACKGROUND AND EXPERIENCE OF ORGANIZERS
Stephanie Rosenbaum
Tec-Ed, Inc.
P.O. Box 1905
Ann Arbor, MI 48106 USA
1-734-995-1010 voice
1-734-995-1025 fax
stephanie@teced.com
Stephanie Rosenbaum is founder and president of Tec-Ed, Inc., a 15-person firm specializing in usability research and information design. Headquartered in Ann Arbor, Michigan, Tec-Ed maintains offices in Palo Alto, California, and Rochester, New York. Tec-Ed clients include Sun Microsystems, Xerox, Cisco Systems, AOL, and a wide variety of smaller firms.
A member of ACM SIGCHI, the Human Factors and Ergonomics Society, and the Usability Professionals' Association, as well as a Fellow and Exemplar of the Society for Technical Communication, Stephanie is a past vice-chair of ACM SIGDOC and headed the STC's Research Grants Committee for five years. Stephanie was awarded a Millennium Medal in 2000 by the IEEE; she recently contributed a chapter to the Copenhagen Business School Press volume, Software Design and Usability. Her research background includes anthropology studies at Columbia University and experimental psychology research for the University of California at Berkeley.
Judith Ramey
University of Washington
14 Loew Hall
PO Box 352195
Seattle, WA 98195-2195 USA
1-206-543-2588 voice
1-206-543-8858 fax
jramey@u.washington.edu
Judith Ramey is professor and chair of the Technical Communication Department at the University of Washington. She edited a special issue of IEEE Transactions on Professional Communication on usability testing in 1989. Judy regularly gives courses, tutorials, and presentations on usability issues at conferences held by ACM SIGCHI, the Society for Technical Communication (of which she is a Fellow) and the Usability Professionals' Association, as well as for corporations.
With Dennis Wixon, Judy edited Field Methods Casebook for Software Design, published in 1996. In cooperation with the University of Twente in the Netherlands (and sponsored by an STC research grant), she organized a workshop that resulted in five sets of heuristics for Web communication, described in the August, 2000 special issue of Technical Communication.
Together, Judy and Stephanie have presented tutorials, workshops, and SIGs at UPA, ACM SIGCHI, HFES, and other conferences for the past ten years, including:
- Current Issues in Assessing and Improving Information Usability (CHI 2000 SIG).
- Corporate Strategy and Usability Research (CHI 1996 Workshop).
- Field Studies: Taking Usability on the Road (UPA 1995 Tutorial).
- Methods for Investigating Usability During Early Product Design (UPA 1993 Tutorial)
Bibliography of Related Publications by the Organizers
1. Boren Ted, and Judith Ramey. "Thinking Aloud: Reconciling Theory and Practice." IEEE Transactions on Professional Communication, 43 special issue on usability research methods, (September 2000): 261-278.
2. Jong, Menno de, and Judith Ramey. "Introduction: New Perspectives on Document Evaluation Methods." IEEE Transactions on Professional Communication 43, special issue on usability research methods, (September 2000): 237-241.
3. Humburg, Judee, Stephanie Rosenbaum, and Judith Ramey. "Corporate Strategy and Usability Research: A New Partnership." CHI 96 Conference Companion (April 1996), 428.
4. Rosenbaum, Stephanie. "Not Just a Hammer: When and How to Employ Multiple Methods in Usability Programs." UPA 2000 Conference Proceedings (June 2000), Tab 19.
5. Rosenbaum, Stephanie, Janice A. Rohn, and Judee Humburg. "A Toolkit for Strategic Usability: Results from Workshops, Panels, and Surveys." CHI 2000 Conference Proceedings (April 2000), 337-344.
|
|