iCurator  Low Fidelity Prototype and User Testing

for Group 5, CS 160, Fall 2001

Kevin Wang: Facilitator

Leila Takayama: Greeter & Computer

Chuck Moidel: Observer

Scott Carter: Observer

URL of this document: http://ratbert.bmrc.berkeley.edu/courseware/cs160/fall01/Projects/Group5/Lowfi.html

Introduction

The iCurator is a system created for adult museum visitors a wide range of experience with art, museums, and technology. The purpose of the iCurator is to give museum visitors a more fulfilling experience at the museum by allowing visitors to automatically retrieve an assortment of information on artworks, send links and information via e-mail, and navigate their way successfully through galleries. By using barcodes placed throughout the museum, the iCurator provides useful information to the visitor based upon the visitor’s location whether it be at the entry of a gallery, at the elevator doors, or in front of a specific artwork.

The purpose of the low fidelity user testing experiment is to have real users evaluate the iCurator system to find usability problems with the current design. By using a low fidelity prototype, it becomes possible to produce many different versions of the iCurator design very quickly so iterations are cheap, quick, and easy. Using comments collected from the users, we aim to decide upon changes to be made and elements to be kept based upon the early-stage iCurator design.

Mission Statement

To create an interactive, contextually aware curator-like system that enhances the experience of museum visitors by providing the means for more easily and fully appreciating the art, delivering a more personalized collection of information about artwork and the museum, and extending the museum experience beyond the walls of the museum. We will achieve these goals by providing navigational aide, easily accessible in-depth information about artworks, the ability to e-mail that information to view later, and the convenience of contextual awareness.

Low Fidelity Prototype Description

The low-fidelity paper prototype produced for the initial design of the iCurator was done on quarter-sheets of 8.5”x11” paper [figure 22]. Although this is larger than the size of a PDA screen, it was easier to make changes to the prototype given more space. To make up for the unrealistic size of the paper “screens”, the page resolution was lower than a typical PDA screen. The prototype was written up in pencil so that changes would be easier to make on the spot. The paper prototype includes screens that are useful to the users for performing the assigned tasks as well as some auxiliary screens just in case they chose to navigate elsewhere through the iCurator system.

In general, the interface of the iCurator is based upon that of an Internet browser such as Internet Explorer or Netscape Navigator. Underlined words denote links. Buttons look like html buttons. Due to the very limited size and resolution of PDAs, iCurator screens are unlike desktop Web browsers in that they have fewer words, smaller images, and more links.

Each iCurator screen follows a few general layout rules. Screen titles are at the top. At the top right corner is the floor at which the user last scanned a barcode. If the user just entered the museum and checked out an iCurator, then it is assumed that the user will be on the 1^st floor. Just below the title are links to parts of the iCurator system that are parents of the node of the current screen. For instance, when a user is viewing the artwork “Yosemite Winter Scene” [figure 9], just below the title are the links “home: 4^th floor: gallery x: Yosemite”. This indicates to the user how deep they are within the schematic tree diagram of the infrastructure of the iCurator system. It also allows the user to go back to any parent page with only one click as opposed to hitting the back button several times over. The center of the page is dedicated to content, including text, links, buttons, scrollbars, textboxes, and images.

At the bottom of every single page is the main menu [figure 21]. It is possible that the pointers to the locations on the main menu will be coded into the hardware of the PDA, but for this low-fidelity prototype, they have been placed on screen. The main menu consists of the choices: search, help, map, e-mail, back (an arrow pointing left), and forward (an arrow pointing right). These menu choices may be selected at any time during the use of the iCurator.

On the pages about specific artworks, a smaller version of each artwork is shown. Below are links to information about the artist and links to video, audio, articles, other works by the artist, and related work to the piece. If there is some portion of the artwork that is particularly interesting, users may click on that portion to get more information on that point of interest. This is demonstrated on the page for “The Bride” in which the rat in the foreground of the image is a point of interest [figure 13].

An icon of an envelope appears in the upper right corner of pages with more information about a specific artwork [figure 14]— users may click this button to e-mail information from those pages to themselves or others for future reading. Each informational page has a very small amount of content written about the piece, then links below to information about the artist, critiques of the artwork, historical context of the creation of the piece, etc.

The search [figure 5], map [figure 2], and e-mail [15, 16] portions of the iCurator are somewhat separate from the rest of the navigation through the museum and artwork information. The search function allows users to search the iCurator’s database for titles, artists, genres, galleries, floors, or keywords. Relevant links are listed and may be viewed by scrolling down the screen. The map of the museum has functional links at various points of the image. For instance, the user may click on the image of the 3^rd floor to get to the page for the 3^rd floor galleries. The e-mail pages include a textbox for e-mail address entry. iCurator automatically fills in the subject and content of the e-mail, but both sections may be edited. To enter text, the user may click on an on-screen keyboard at the bottom of the page. Before sending the e-mail, the user is asked to confirm the e-mail address and content before proceeding.

Because the user testing was not performed at an actual museum, we pretended to be at the Berkeley Art Museum. Using information from the BAM Web site, we obtained content about specific artworks that are actually located there, maps of the museum, and printed images to pretend to be real artwork in the user testing setting.

Method

The participants in our experiment were two females, CVB and SCH, and one male, KK. All participants were in their twenties and were friends of acquaintances that were willing to participate in our usability tests. Both female participants had very limited experience with PDAs or pen input devices, whereas the male had slightly more experience. All of the users, however, were very familiar with computers and Internet applications, such as sending e-mail and navigating Web browsers.

The usability testing took place in the apartment of group member Leila Takayama. During each test the user was seated in a chair at a large table with the low fidelity paper prototype set in front of them and a tape recorder set off to the side. They were given a pencil to use as a pointing device and shown a palm pilot to indicate the size and type of device that the interface would be displayed on if they were actually walking through a museum. They were also shown a picture of art with a barcodes sketched underneath to demonstrate how an art exhibits would appear in the museum.

Leila Takayama greeted each participant at the start of every test and had them fill out the consent form and the pre-questionnaire. Next the user was given the demonstration script and asked to read it aloud. Kevin Wang, the facilitator, would then ask the user if they had any questions about the content of the demonstration script and Leila, the “computer,” would set out the first screen of the iCurator prototype. For each task the participant would read aloud the task description and then use the iCurator until they felt the task was finished. During the user test Charles Moidel transcribed important dialogue and events on a laptop, while Scott Carter took notes of observations and made sketches by hand. At the end of each task, the user was asked to share their opinions about the ease of the task, the usability and content of each interface screen, and their likes and dislikes about the application. After the user completed all three tasks, Kevin thanked the participant and asked them to honestly fill out the post-questionnaire at the table in the adjacent room.

The three user tasks were as follows:

1) A friend of yours mentioned an interesting piece of art on the fourth floor at the Berkeley Art Museum so you went to check it out. You have just entered the museum and want to find your way to the fourth floor. Please use the iCurator to navigate to your destination.

2) Now that you are on the fourth floor, you are looking for the work called “Yosemite Winter Scene.” Please use the iCurator to find information about that piece of art. Once you find that piece, please look for more information about the gallery in which the piece resides.

3) After wandering around the museum you end up on the first floor of the museum and are standing in front of a piece of art called “The Bride” by Joan Brown. You would like to read more in-depth information about the piece after leaving Berkeley Art Museum. Please use the iCurator to choose an article about “The Bride.” Then send that information to yourself via e-mail.

During each experiment we made an audio recording and kept detailed records so that we could later determine the length of time each task took and the number and type of errors that were made. We also made note of the parts of the interface that were very easy for the users to understand and the parts of the interface that were considered cryptic. The user’s opinions and answers to the post-questionnaires have given us a great amount of feedback as to what the weak and problem areas are of the iCurator interface.

Results

We encountered difficulties testing the device out-of-context but did try to help the participants as much as we could to compensate for the lack of physical presence. However, the final product will need to be tested in a museum setting to ensure its effectiveness.

The first problem we encountered was related to the disconnect between the map and the physical space of the museum. Perhaps if the user were physically in the museum this problem would automatically be remedied. Furthermore, it became evident that floor labels should be underlined and linked as a cue to the user to click on the link a la IE or Netscape.

The second difficulty we observed was that each floor of the BAM is a gallery whereas most museums have multiple galleries per floor; the users were confused because the map of the floor showed only an underlined link to one gallery.

We also could not simulate the bar codes that are supposed to be generously placed throughout the museum, and thus most participants did not consider using bar codes until the last task. The bar codes were thus vastly underused and helped user navigation only minimally.

Users were able to view articles and send e-mails relatively easily. The search function was overused, most likely because of the lack of barcodes. But the search function worked well, and in each case it was used the user found what he or she was searching for. The e-mailing function was also easy to use and the only suggestion users had was that, if they decided to e-mail multiple articles, the system remember their e-mail address so they would not have to re-enter it. Also, some users proposed that after sending an e-mail they be presented with a confirmation screen instead of returning to the previous screen, the default action. Navigation through the content seemed relatively easy for users; we suspect that it will easier still with a real device (with a color screen and barcode reader) and without the attention of four group members.

Discussion

This experiment shed light on sundry major and minor design flaws. In the worst case, in which the first user got stuck immediately on the first task, we were forced to change the interface right away. In general, though, users were able to navigate to content rapidly and use the search and e-mail features with ease. We also concluded that a “quick start” screen would be useful and that we were unable to adequately test the context-awareness system.

The first task requires the user to use iCurator to help them navigate to the fourth floor. However, in our first paper prototype the floors were not labeled on the museum map. As a result of this error the first user was unable to complete the task, and we added floor labels to the map and a graphic to each interior page indicating the floor on which the user last scanned an item. In the process of redrawing the map, we realized that we had left out restroom labels, which we will add in later prototypes.

Users had little difficulty navigating to pertinent content. However, some of our links were not descriptive enough and should be elaborated upon in later versions (e.g. “video” should become “a video of an interview with the artist”). Also problematic was the lack of a home button on every page. To remedy this we will remove the map button and replace it with a map/home hybrid button (on iCurator, a map of the museum appears on the home page). In addition, although present on every screen, participants never made use of the help button. As a remedy, we may enlarge the button in later prototypes. Finally, we found that the search feature adds an important dimension to the navigability of the system and therefore must be included in future prototypes.

iCurator’s most significant success was the e-mail feature. Users seemed excited by the possibility that they could e-mail themselves information, investigating it further asynchronously. Given this enthusiasm, it is possible that iCurator may act as a node through which information passes rather than an end in itself, similar to David Hilbert’s and Jonathan Trevor’s m-Links system. The e-mail feature was not without its problems, though — it originally lacked a confirmation screen, which we added during the experiment, and users encountered difficulties entering special characters (e.g. “@”). Also, users were sometimes confused about the nature of the content in the e-mail and seemed frustrated that they would have to re-enter their e-mail address if they wanted to send multiple e-mails. However, we suspect that both issues will be remedied in later prototypes wherein iCurator will automatically generate the subject, content and, if possible, recipient address.

Also, it quickly became evident that users would benefit from an introduction to using the system. It was less clear, though, whether users needed an introduction to using handheld devices or simply an introduction to navigating through this specific low fidelity prototype.

Finally, as stated in the previous section, we were unable to adequately test the barcode scanning section. There was no way to avoid this problem — without context it is difficult to simulate context-awareness. We hope to test this aspect of the system soon after deploying our first high fidelity prototype.

Consent Form

for participation in user testing of an iCurator prototype

This user study is being conducted as part of a computer science course, CS 160. The aim of the user testing is to improve the current design of the iCurator prototype. Data will be collected by interview, observation and questionnaires. Participation in this experiment is voluntary. Participants may withdraw themselves and their data at any time without fear of consequences. Any concerns about the user test may be discussed with the project members (Scott Carter, Chuck Moidel, Kevin Wang, Leila Takayama). Participant confidentiality will be provided by the separate records of names from data. Data will only be identifiable by participant codes. No identifying information about the participants will be available to anyone except the researchers and their supervisors. To make sure we have a correct record of you comments we may be taking a voice recording of the interview.

There are no foreseeable risks to you from participating in this user testing. There will be no costs to you, other than your time involved and any personal transportation costs.

It is expected that one interview will be sufficient. If we need follow-up information we will contact you by phone or email.

I hereby acknowledge that I have been given an opportunity to ask questions about the nature of the user test and my participation in it. I give my consent to have data collected on my behavior and opinions in relation to this user test. I understand I may withdraw my permission and data at any time.

Name ______________________________________________

Date _______________________________________________

Signature____________________________________________

Pre-questionnaire for iCurator prototype user testing

Identification Code: ________________

Date: ___________

Age under 18 18-20 20-25 30-35 35+

Gender male female

Highest Academic Degree □ high school diploma or GED

□ bachelor degree

□ graduate degree

□ none of the above

□ other ________________________

Academic or Occupational Field: __________________________

How many times have you been to a museum in the last year?

0 1 2 3 4 Other ______

Have you ever gone in a curator-guided tour at a museum? Yes No

If yes, where? ___________________________________________________________

When you visit museums, how often do you go on curator-guided tours?

□every time □every other time □occasionally □rarely □never

Have you ever used audio (or other media) tours at a museum? Yes No

If yes, then where did you do the tour? _________________________

When you visit museums, how often do you go on audio-guided tours (or other media-based tours)?

□every time □every other time □occasionally □rarely □never

Do you own a PDA? Yes No

If yes, when did you purchase it? _________________________

How often do you use it?

□daily □weekly □every few weeks □monthly □other _____

What do you use your PDA for? ___________________________________________

User Testing Incidents

User Test #1 - CVB

Task #1

I don’t have any idea what to do? I guess I’d go to the map page.

Oh, I didn’t even see map there. So I’d click on map

I don’t have any idea what to do?

How do I get to the 4^th floor? There aren’t any labels on the map!

Still lost! (Can’t tell what floor you’re on!!!)

Not sure what to do. (We tell her that she can click on the screen anywhere.)

If I knew it was the fourth floor, I’d go there. So when I’m there, I’d guess I’m done.

“Shoot me now”

Task #2

Ok so I’m on the 4^th floor looking for Y W Scene

I’d guess I’d search…by title … Y. I click on it.

I don’t understand what any of this is.

Don’t know when the task done.

Task #3

But I’m on the 4^th floor. Ok, I walk downstairs.

I’d go home. Hmmmm So I’d search for the piece.

Kevin: you know there’s a bar code underneath the painting.

I guess I would read articles

The envelope is e-mail supposedly

So how do I type my e-mail address?

User Test #2 - KK

Task #1

I walk into the lobby, past the 1^st floor, click on the 4^th floor of the iCurator

(Looks Shocked)

Click on Gallery X… [and gets list of pictures before it is decided that the task has been completed.]

Task #2

Click gallery X, says home 4^th floor. Click on Yosemite winter Scene. List of artists, click on read articles. Up comes articles. Would read up on the artists life. But its not yet implemented. CLICK ON MORE. So I want to go back a page. Click on the backward arrow. Click on video. Watch video. Click on audio. Click on back again.

Specify that specifics are links to other page.

[Or, more explicitly, suggests that we should include relevant titles for audio and video files.]

Task #3

Click back again. I’ll press map. Will that take me back to the beginning? Click on 1^st floor. I’m on the first floor now. I’ll click on gallery Y right now. Maybe it is not so obvious that clicking on gallery Y shows you pictures in the gallery

Oh yeah I am standing in front of the picture. I use the scanner on the side of the PDA to scan the picture that is in front of me.

Press the E-mail button at the bottom of the page. Use QWERTY keyboard to type in e-mail address. (Missing @ key) The e-mail thing is kind of ambiguous. Can I e-mail my friends from here? Is there a home button? I’d put the HOME button on the toolbar. (Worried about being spammed)

[Other comments… perhaps we should combine the home and map buttons, and how are we going to deal with #s and the @ symbol on the meager PDA keyboard?]

User test #3 - SCH

Task #1

I’d go strait and go up the curved stair-well. Just by the map I could find it. Hard to indicate elevation due to lack of detail. Colors could help. Indicator on the side to denote rise in the floors. I’d also indicate stairs.

Task #2

Hit the 4^th floor. That would be my first reaction. Now I’m on the 4^th floor. OK….

I wouldn’t know were to start with the piece of art. A list by title or artist would help. Split the gallery into cubism and minimalism.

Leila: What if this was a link?

Scot: Feel free to explore and make mistakes

Kevin: Click on random things.

I’d click on cubism. Under construction. Go back

Minimalism. Under construction. Go back

Is “Gallery X” the same as “X gallery”?

Ok I’d pick piece B. Under construction. Go back. Piece A. under construction

Click on Yosemite winter scene. Trying to decide what I’d do next to get the most information quickly. I’d probably go to articles.

Click on video of interview.

Kevin: Blah blah blah

I really like this e-mail thing up here, so that I could remember it. I’m going to click on this e-mail thing up here. Would I have a choice about e-mailing? What am I going to e-mail?

Cool. I’m going back to here. THE TASK IS FINISHED

It needs a brief introduction to this article instead of having to pick between video, audio, etc. It’s really navigable. I could go places easily.

[So, to generalize from the above comments and this one, perhaps for each artwork we should have an introductory sentence and more pertinent descriptions for links to video and audio content]

Task #3

How would I get home? Click on home. I’d click on the first floor. Would I have to click on the first floor or could I just scan the barcode?

I’m at the bride in gallery Y. I’d go to read articles and then I’d …. Next step would be to go e-mail myself. I’m sending it to myself.

I’d look for a button that says send instead of confirm

Leila: What would have made that easier?

It would be nice if it prompted me for my e-mail only once at the beginning?

As long as I don’t have to repeatedly input my e-mail address.

I would put a restroom map somewhere that was easy to find.

Critical Incidents with Severity Ratings

Negative incidents regular font

Positive incidents italicized

Critical Incidents	Scott’s SR	Leila’s SR	Chuck’s SR
No floor # labels on map	4	4	4
No confirmation screen after sending e-mail	2	2	2
On artwork pages, links to audio and video content are not descriptive enough	2	2	2
No home button on main menu bar	3	3	3
Unable to easily type characters to send e-mail (@, ., etc.)	1	1	2
Map button on main menu bar not very visible	1	2	2
E-mail icon is easily recognizable	-	-	-
Users forget that barcodes are there (could be a problem with the lack of context rather than an interface problem)	2	3	2
User unclear about exactly what is being e-mailed	3	3	3
On artwork pages, user wants brief intro to the article	1	2	2
On e-mail confirmation page, user wants the button to say “e-mail” instead of “confirm”	1	2	2
User wants iCurator to “remember” e-mail address	2	3	2
User wants restrooms on the map	2	2	2
Users do not notice “help” button or link	3	3	2