Thumbnail: Tema Frank

By Clifford Anderson
Senior Usability Engineer
Wachovia Corp., Charlotte, North Carolina
clifford.anderson@wachovia.com

Cliff Anderson is a Senior Usability Engineer at Wachovia, the fourth largest bank in the US. He has been doing usability work for almost 20 years.

Tema Frank’s approach to usability is as unusual as her name. (“It rhymes with Emma,” she notes. “It was my grandmother’s name.”) That distinctive approach is also reflected in the name of her company, Web Mystery Shoppers.

Web Mystery Shoppers Company
Mystery shoppers are from the world of traditional marketing, a field where Tema has 25 years’ experience. That experience also includes an MBA, as well as running “hundreds of focus groups.”

A mystery shopper goes into a store, bank, or restaurant, and transacts some business. Unknown to the employees, the mystery shopper files away that experience in their head, then immediately sits down to answer a series of questions and write a report on it. Not only does this keep the employees on their toes, it allows the company contracting for the service to see what works and doesn’t work relative to customer service, store design, and other important factors.

Tema thought the idea of mystery shopping could be just as valuable for visits to the web. Her mystery shoppers log in to her site from their home, are directed to specific client sites, and are given tasks to complete. While there, they answer questions as they go, “page by page.” They may also be asked to contact customer service to get a feeling for the complete user experience.

Potential shoppers are screened by completing a sample assignment and by answering questions about their demographics. To make sure they do not become “professional testers,” those who are accepted never stay with the company after a handful of studies. Tema has approximately 50,000 shoppers at any one time.

Online Shopping Approach
One advantage of doing mystery shopping online is that traditional in-person mystery shopping places “way too much weight on one visitor to one store at one time.” Her company has at least 30 people test each website.

Tema believes this approach also has some advantages over traditional usability testing as well. Though she realizes “this is a bit controversial,” one of her goals in starting her company was to bring “a little more statistical validity to the way things are done. It really helps you with priorities. If you’ve got 5 testers and only one of them comes across the problem, surely if it’s a catastrophic problem, you’ve got to deal with it. But there are a lot of mid-range problems, where you really don’t know how common or important they are if you only have a few testers.”

Tema also points to individual preferences, something that often comes up in traditional testing, like it or not: “If 85% of your testers say, ‘I hate this color,’ then you’ve got to do something about it. But with preferences, you’re always going to get a range. Large sample sizes give you a feel for how common the reaction is, and how strong it is.”

Abandoning the Lab
In abandoning the lab, Tema also cites the many different settings users access their computers from, as well as the different hardware users actually own – “their buggy home computers,” as she calls them. In particular, she cites her first study with the Royal Bank of Canada (RBC). This one involved 100 users (though she has also done studies with over 200 users). “Because we had a large number of testers, we were able to find problems that others had missed,” she notes. “People on certain platforms were getting way more error messages than other people, and that’s something the bank hadn’t identified before.”

She also thinks the lab setting is somewhat unrealistic, noting that users “are not testing at their own computers. It isn’t their home, their office. It’s an unfamiliar situation and environment.”

Tema recognizes that her approach “has strengths and weaknesses” though. She sees the “biggest strength of the lab is that you can ask follow-up questions. With us, we’ll be reading the reports, and occasionally they’ll say something that doesn’t quite make sense to us. We just have to say ‘hmm,’ and leave it that.”

“Another advantage to the lab is that you can have the muckety-mucks in your company sit behind the mirror, or watch the video, which can be quite powerful,” she points out. Reflecting, she confidently states that “both types of testing have value.”

Getting Interested in Usability
Tema’s interest in usability – and the idea for her company – came from her own experience as an end user on the web: “I really got interested in usability when I was trying to do some shopping, back in the fairly early days, and was just frustrated. I started doing some research, and discovered there was this field called usability.”

It was a natural match. “I’ve always been very systems-focused,” she points out, “trying to make systems work well with minimum human error. I was interested in psychology, interested in business, interested in business processes.” Tema became interested and well-versed enough in usability to start Web Mystery Shoppers in 2001.

That first customer of hers, RBC, pointed Tema in several directions she would continue to pursue. One was her interest in financial services. “Usability is crucial in financial services,” she points out. “What you’re dealing with is a very important thing to people, a very personal thing.” She also cites security and privacy issues, as well as the early success of online banking, noting that “the financial reasons for banks going online were so compelling.” Other specialties include the travel industry, government, and florists. “It’s been quite a range,” she notes.

Other Experience
That experience with RBC was also her first with cross-channel testing: “We had our testers call up customer service, and some of the call centre staff didn’t know the website, so they’d say, ‘It’s on the website,’ but they wouldn’t know where. We were trying to look at the total picture.”

Tema has done some traditional usability testing as well, especially with the visually-impaired, an experience she calls “a real eye-opener.” In particular, she notes that the “problems that they have are so easy to fix,” citing the use of “More” links in particular. For visually impaired users who go through lists of links to process a page (a very common strategy), those “More” links are totally without context. “More about what?” Tema asks.

In addition to keeping very active with the UPA (where she is on the editorial board for User Experience magazine), Tema has also done important work with the Web Analytics Association and the Web Marketing Association. She also teaches a course in electronic marketing at the University of Alberta.

Those activities point to a particular interest of hers and a defining experience in her career, the intersection of marketing and usability. Tema sums it up well: “For a company to be both efficient and effective, marketing and usability people should work very closely together, to ensure that the right products get designed, the right way, and then marketed effectively.”