The U P A voice
August 2005 Contents

Featured UPA Links:

Job Bank

Member Benefits:
Discounts and Special Offers:

Event Discounts
Order your usability books now from the link from this site. Part of all sales help fund UPA.

Thumbnail: Jared Spool

by Cliff Anderson
Senior Usability Engineer
Wachovia Corp., Charlotte, North Carolina

Jared SpoolJared Spool is usability’s “bad boy.” With his challenging opinions and his theatrical way of presenting them, he has excited and frustrated usability practitioners and thought leaders for many years. Agree or disagree, love him or hate him, you have to give him credit. Jared’s is one of the most easily-recognized names in the field.

Jared originally came to usability from the technical side of the business. He started out writing code for office software at Digital, once the number-two computer company in the world.

Digital was one of the original hotbeds of usability. The need for something like usability – “This was before anybody used the term,” Jared points out – arose with a market shift from technical users to the average person. Digital responded by putting together an “all-star group” of psychologists and sociologists. Except for a few other standouts like IBM, this was something that “no one had really ever done before.”

For Jared, it was a perfect match: “They didn’t know very much about engineering, and I didn’t know about psychology, so we sort of made a deal. I found it absolutely fascinating what they were doing, and they seemed to think learning something about engineering was a good idea. We worked together to try and identify how to develop applications with the user in mind, to understand what would make an interface that we now refer to as ‘intuitive.’”

From Digital, Jared moved on to other companies and positions, sometimes more technical and sometimes more usability-oriented. In 1988, he started his own company, Usability Interface Engineering, or UIE.

Jared jokes that the only reason he started UIE is “because I couldn’t keep a day job” and that “I don’t score well in ‘plays well with others’.” In fact, UIE was one of the first in the field and is still going strong. They will celebrate their 17th anniversary in August.

The early days of UIE involved quite a bit of evangelization. One of the presentations that Jared showed to potential clients was called “Software That Makes People Cry,” and was based on a test that UIE had run where a user did just that. He originally thought companies would “flock to do [usability testing],” but found he had to sell the idea and to tailor his message.

It was an important lesson in a number of ways. For one, Jared learned that “technology goes through stages, and at different stages, usability is a different problem.” Many of the companies Jared spoke to originally were “at the ‘talking horse’ stage, where people are just trying to get product out the door.”

Addressing the client’s real business needs was another lesson: “You have to talk to what the issues are. Once we realized that, everything changed.” Currently, Jared points out, “every recommendation we make we have to pinpoint to a business objective, usually something to do with increased revenue, or some other activity obviously tied to the success of the business. If we see something now that is frustrating users, but we can’t figure out how it’s affecting business, we don’t report it.”

As UIE grew and tested more and more users, Jared noticed that “we kept seeing the same problems over and over again.” To address these issues, UIE moved into more of an educational role. Jared started a newsletter, called Eye for Design. Workshops at UPA and CHI eventually evolved into UIE’s own conference, the User Interface Conference.

These conferences didn’t simply provide a venue for Jared and UIE’s research. To explain what they were finding, Jared brought in the experts, people like Jakob Nielsen, Alan Cooper, and Karen Holtzblatt. This years’ conference, in October, marks the event’s 10th anniversary.

Lately, Jared has been turning his eye to usability as a whole. For motivation, he points, in particular, to studies like Rolf Molich’s CUE, which he calls “really telling.” This study looks primarily at heuristic evaluations, finding that there is “very little overlap of the problems that [different teams] found.”

Jared found something similar in UIE’s own work. Contrasting evaluations of systems directly with tests of the same systems, he found they “were really bad at predicting problems,” with a less than 50% accuracy test. “We were less accurate than a coin flip,” he notes.

Overall, Jared believes that “we don’t have a reliable way to find serious problems. If the client had wanted to find all those problems, he would have had to hire all 17 teams [in the latest CUE study]. It puts into question everything we promise usability to be, which is a way to identify problems.”

Jared also points to issues with testing. For example, he found that the wording of a task can have a major influence on the results: “We learned early on that very simple changes in wording result in completely different user behaviors.”

He also points to other UIE studies whose counter-intuitive findings question usability maxims like the importance of download time (Jared found users’ perceptions were strongly colored by whether they completed their tasks), number of users (he believes you often need a lot more than seven), and user motivation (on tests of e-commerce sites, UIE gives users money and simply tells them to spend it).

“We’re going around promising people that usability will help you improve things when, in fact, we don’t really know if that’s true,” he states. In typically blunt fashion, Jared believes that, overall, “nobody really knows what we’re doing, myself included.”

He compares the field to medicine in the early 19th century: “In the early 1800s, medical practice was really hit-and-miss. What doctors did and what they were trained to do sometimes worked and sometimes made the patient worse, and they didn’t have any way of knowing what they did.” Jared deadpans, “A lot of usability practice today is a lot like blood-letting.”

So what’s the solution? “We need to make a huge investment in usability science,” Jared counsels. “We need some major theoretical underpinning to explain how people interact with design and to explain good design. I think if we push hard, we can come up with that.”

This is, in fact, the direction that UIE has been heading in the last few years. “We realized that there were so many things we don’t know how to do,” says Jared. “We realized that there were many questions that our clients had that nobody knew the answer to. So we started to put research together, to actually start and answer some of these questions. For the past four or five years, we have been very focused on researching some very large problems.”

“We are no longer a consulting company,” reflects Jared. “We are a research company. Consulting represents a very small percentage of our revenue base. And the only consulting we do is consulting that’s basically funded research.”

“We see ourselves as serving the usability community as a whole,” he continues. “Our goal is to completely eliminate frustration with technology. We see that as a hundred-year mission. I’m building a company that’s going to be around long after I’m gone.”

In the meantime, Jared does confess to being something of a rebel: “I think to some extent I am, because I’m asking really hard questions. I’m asking why.”

  Usability Professionals' Association
140 N. Bloomingdale Road
Bloomingdale, IL 60108-1017
Tel: +1.630.980.4997
Fax: +1.630.351.8490
Contact the Voice