upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Text Advertising Blindness: The New Banner Blindness?

Justin W. Owens, Barbara S. Chaparro, and Evan M. Palmer

Journal of Usability Studies, Volume 6, Issue 3, May 2011, pp. 172 - 197

Article Contents


Methods

The following sections discuss the participants, materials, and procedures used in this study.

Participants

Thirty-two participants were recruited from psychology and general education courses at Wichita State University for course credit, and all gave informed consent before the experiment. Seven participants were excluded from the study due to problems with eye-tracking calibration. Twenty-five participants (10 males and 15 females, mean age 23.5 years, SD 6.17 years) had Internet and computer experience, with the majority reporting at least 7-14 hours of computer use per week.

Participants reported using the Internet for a variety of reasons, the primary three being educational purposes, e-mail, and entertainment. Other reported activity included gathering product information, researching travel information, work/business, personal needs, browsing the web, and communicating with others.

Materials

The following sections discuss website stimuli, tasks, eye-tracking apparatus, and questionnaires.

Website stimulus

A website was created for the study consisting of 29 web pages organized into seven categories. The overall theme of the website was to provide information about travelling to Hawaii and its six major islands. The website categories were the Hawaiian Islands, the Island of Maui, the Island of Molokai, the Island of Oahu, the Big Island of Hawaii, the Island of Kauai, and the Island of Lanai. The sub web pages for each island were organized into history, points of interest, travel tips, and a summary page. The category for the entire chain of islands included similar topics such as travel tips, festivals, and state history.

Each page of the website had a similar organizational structure. All the pages contained a top navigation bar that listed each island and a Home section with information about all of the Hawaiian Islands. Left-side navigation contained sub-category web pages for each of the main categories. The center of the web page consisted of a content section that included two text advertisements at the top, a section header, copy, and a single photo related to the category or sub-category of the web page. Immediately to the right of the center area was a list of 5-6 text advertisements. The bottom of the page contained a single footer with copyright information. Above the main navigation was a website logo. An example web page from the website is provided in Figure 2.

The majority of the content was adapted from the Hawaiian Convention and Travel Bureau's website (http://www.hvcb.org/). Some content was adapted from Wikipedia articles about the history and festivals in the Hawaiian Islands. The content was edited to ensure it would proceed beyond the fold of the web page, thus requiring users to scroll to see the entire content.

The text advertisements were generated from Google.com searches for Hawaiian travel related topics. In total, 103 unique text advertisements were selected from the Google.com searches and were customized for each of the categories and subsequent sub-categories in the website. With customizations of the text advertisements, 205 text advertisements were created and placed within the 29 web pages of the travel website.

Figure 2

Figure 2. A page from the Hawaiian travel web page stimuli

To analyze the eye-tracking data, Areas of Interest (AOIs) were defined for each target page. AOIs are geometric areas of a web page that correspond to specific content. All AOIs excluded any graphical components. The target locations were within their respective AOIs (e.g., Content target locations were located within the Content AOI; see Figure 3). These AOIs were all located above the initial page fold, or point that requires scrolling to view content or web elements below.

Figure 3

Figure 3. An example of a web page with the Content AOI, Top Ad AOI, and Side Ad AOI, the target was always located above the page fold.

Tasks

Two types of search tasks were used in the study: exact and semantic searches. Exact searches directed participants to find an exact target in the website. For example, participants were given the altitude of a city (i.e., nearly 1,700 feet) and asked to locate the fact in the website. Semantic searches instructed participants to find a target within the website with information semantically related to the goal. For instance, participants were asked to find information about the type of military branch located at the National Landmark in Oahu that was dedicated to the attack that led the U.S. into World War II. Target locations were equally likely to appear within the content of the page or within a text advertisement. For targets appearing inside a text advertisement, half were located in the top ad area and half were located in the side ad area.

Overall, participants completed 16 target-present search trials: eight target locations in the Content section, four target locations in the Top Ad section, and four target locations within the right Side Ad section. In each of the three sections, half the tasks were exact searches and the other half were semantic searches. Additionally, participants performed one target-absent search where they were asked to search for a target that was absent from the website, but could fall logically under a section of the website. See Table 1 for examples of the tasks used in the study.

Within the content of the page, the target location was positioned either in the first or second paragraph, which were always located above the page fold. In Figure 3, these targets were located in the region called the Content AOI. In the top ads, the target location was situated either in the first or second advertisement (Top Ad AOI). In the right ads, the target location was positioned in the top or bottom half of the advertisement section (right Side Ad AOI). The target locations were equally likely to be in the top or bottom half of each section. Targets were spread across all seven categories of the website. Tasks for each target location were worded similarly to minimize indication of the target location. Task order was counterbalanced across participants.

Two practice trials, one semantic search and one exact search, were provided to allow the participants to familiarize themselves with the search task types. These practice trials were not analyzed. The targets for both practice trials were located within the center content section of the web page.

Table 1. Examples of task scenarios used in the study. For exact searches, participants were instructed that they should search the website for the bold, italicized text found in the task scenarios.

Table 1

Eye-tracking apparatus

A Pentium IV-based PC with a 96 dpi 17” monitor with a resolution of 1280 by 1024 pixels integrated with the Tobii 1750 eye-tracking system running at 50Hz was used to capture eye-tracking measures. Tobii StudioTM software was used to record and provide fixation and AOI eye-tracking data of the participants. Custom software was created to analyze and generate summary statistics from the eye-tracking data collected from Tobii StudioTM.

Questionnaires

A background questionnaire was used to collect demographic information and Internet/computer experience. A modified version of the System Usability Scale (SUS) was administered to measure user satisfaction with the website (Brooke, 1996). The modification of the SUS focused the usability measures toward websites instead of "system" in order to reduce confusion among participants.

A post experiment questionnaire collected information about ad recall, ad location, and search strategies for the exact and semantic tasks. Additionally, participants were asked if any information on the website was difficult to find in order to gauge if participants associated text advertising with difficult tasks. Several distracter questions were included to prevent participants from determining the nature of the study. These questions asked about color schemes, content, and the appearance of the top and left navigation menus.

At the end of each trial, participants were asked to rate the difficulty of the given task. A 5-point Likert scale was used to measure difficulty (1 = very easy, 5 = very difficult).

Procedure

Participants each completed eight semantic searches, eight exact searches, and one target‑absent search in random order. The study was portrayed and advertised as a website evaluation study to prevent participants from determining the actual purpose. Each experimental session lasted approximately 45-60 minutes. Participants answered the demographics/experience questionnaire and were calibrated on the Tobii 1750 eye-tracker using a nine-point calibration process. If the participant was not able to calibrate accurately, they were dismissed from the study.

To complete the task portion of the study, participants were handed a task card and asked to read it aloud. Participants were allowed to ask questions to clarify the nature of the task before starting, but were not provided information on the target location. Once participants indicated that the task was understood, they handed the task card back to the researcher. This was done to eliminate the possibility of looking at the card while completing the task and thus disrupting their eye gaze from the monitor. If participants needed the task information again during the trial to remind them of their target, they asked the researcher to reread the task aloud. The researcher did not confirm or deny whether the participant's target choice was correct. Participants were asked not to engage in talk-aloud protocol once the trial began to prevent interference with the data collection of eye movements by the eye tracker.

All participants completed the practice trials first. The order of the remainder of the tasks was counterbalanced across participants. All trials were limited to a maximum duration of 105 seconds (determined through pilot testing as a reasonable amount of time to find the target). Typically, this resulted in trial duration of 101 to 103 seconds given Tobii Studio'sTM brief delay in launching the web stimuli. The delay was determined by examining task duration when Tobii StudioTM closed the web stimuli in unsuccessful trials. The beginning of the trial started when the participant clicked a start button. Typically 2-4 seconds later, the web stimuli would appear. In the subsequent analyses, this delay was removed from the task duration. Participants clicked on the search target once they found it and then pressed F10 on the keyboard to end the trial. Eye-tracking data was collected during all trials. If the participant was not able to locate the target and the duration limit was reached, Tobii StudioTM closed the web stimulus and the trial ended. Alternatively, the participant could give up and end the trial early by pressing F10.

After the trial was completed, participants were asked to provide a task difficulty rating using the 5-point Likert scale. In addition, participants were asked to recall the overall web page design used within the website and sketch a wireframe representation on paper of the pages they visited during their tasks. They were instructed to label each section. In some cases, they were asked to provide additional clarification in their sketch if the details or labels of web elements were vague. Finally, participants completed the SUS and post-experiment questionnaire.

 

Previous | Next