Adapting Web 1.0 Evaluation Techniques for E-Government in Second Life
Alla Keselman, Victor Cid, Matthew Perry, Claude Steinberg, Fred B. Wood, and Elliot R. Siegel
Journal of Usability Studies, Volume 6, Issue 4, August 2011, pp. 204 - 225
Article Contents
Practical Framework: The Four Dimensions of Web Evaluation of Information Products
In “A Practical Approach to E-Government Web Evaluation,” NLM suggested a practical framework for performing evaluations of government Web sites (Wood et al., 2003). The article presented practical methods, products, and services that can be used to perform such evaluations. This evaluation framework has been successfully used by NLM and other government agencies over the years on a number of government Web sites, although some of the services and tools have changed over time as technologies and the IT industry have evolved. The framework focused on methods in four evaluation dimensions:
- Usability: behavioral feedback from a limited number of users regarding their experience with the Web site
- User feedback: direct, usually qualitative feedback from actual Web users, typically in survey or focus groups format
- Usage: quantitative data about Web usage levels, primarily from Web log analysis
- Web performance: measures related to the Web site’s response time and availability
The article made a distinction between evaluation dimensions (e.g., usability, user feedback), measures (or variables that can be assessed, such as user satisfaction), and methods (or techniques for obtaining the measures, such as surveys). Figure 3 outlines the four Web evaluation dimensions, along with specific measures within each dimension, as well as analogous areas and evaluation dimensions in VWs.
While the remainder of this paper treats these evaluation dimensions as independent, we do this with the understanding that their boundaries are permeable. As any categorization scheme, this framework is a simplification that reduces a complex universe to a small number of discreet categories, in order to make it more manageable. In reality, usability, usage, and performance can be viewed as outcomes that influence one another (e.g., efficient performance contributes to usability and usability increases usage). However, as evaluation domains, the four outlined areas are characterized by distinctive, if not completely independent, sets of analytic methods. While we recognize that the chosen framework division is imperfect, we find it practically useful for developing and adapting methodologies.
Figure 3. The four Web evaluation dimensions used on NLM’s evaluation framework and their VW analogue.