upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Unexpected Complexity in a Traditional Usability Study

Tharon W. Howard

Journal of Usability Studies, Volume 3, Issue 4, August 2008, pp. 189-205

Article Contents


This study provides empirical evidence that validates Redish's observation that "Ease of use-what we typically focus on in usability testing-is critical but not sufficient for any product" (2007, p. 104). The ease of use indicators in this study, both quantitative and qualitative, suggested that our users found the prototype handbook and its visuals to be an "easy-to-use solution" to their perceived needs. And our client would very likely have been entirely satisfied with the ease of use data we provided. However, had the study's design solely focused on what the client told us they wanted, we would have failed both our client and the users. The true complexity of the tasks would not have been revealed, the lack of "utility" in the handbooks would not have manifested itself (Redish 2007), and our clients would never have seen the need to "think outside the box" of traditional handbook design. They would have lacked the cognitive dissonance that motivated at least some of the creative thinking that ultimately led to the successful handbook they ultimately published.

Fortunately, because we were working with clients who trusted us not to run up the cost of our study needlessly, we introduced some additional elements to the research methodology used. As a result of these additional elements, we observed the following points:

As the field of usability studies grows and usability professionals are required to work with and adapt to the needs of new clients in new fields, the likelihood of encountering complex problems masquerading as simple ones will increase. Our research methods need to accommodate this and ensure that the studies we design for new clients include content experts' assessment of the quality of the users' performance and, when possible, head-to-head comparisons with competing products.


The author would like to thank Wendy Howard who collaborated on the data collection and analysis portions of this study. Thanks also belong to Alicia Hatter, Ginny Redish, and Barbara Mirel for the insightful feedback that was critical in producing the final version of this article.

Previous | Next