PilotStudy-Group:!Xobile

From CS 160 Fall 2008

Contents 1 Group Members 2 Documents 3 Demo Script/Procedure 4 Test Measures

Group Members

• Pilot Study (Mu-Qing Jing)
• Pilot Study (Geoffrey Lee)
• Pilot Study (Perry Lee)
• Pilot Study (Shaharyar Muzaffar)
• Pilot Study (Bing Wang)

Documents

Consent Form (Pilot Usability Study)

Demo Script/Procedure

1. Introduction
   1. Introduce yourself to help relax the subject.
   2. Explain the game, detailing the overall premise and goal of it.
   3. Explain the purpose of this study (e.g., to identify any usability issues that may necessitate a design change).
   4. Show the participant the consent form, giving them time to read it. Re-iterate that he/she can quit at any time.
   5. After the participant has signed the consent form, ask him/her for his/her demographic information:
      1. age
      2. sex
      3. education level and major(s) if appropriate
      4. frequency of playing games, and if he/she plays games, which types of games does he/she play?
      5. level of financial knowledge, how knowledgeable does he/she thinks he/she is?
      6. has he/she planned for retirement?
         1. if yes, how has he/she planned for retirement?
         2. if not, when does he/she plan to start planning?
   6. Explain how the pilot study will be conducted.
      1. Note that he/she will be given three in-game tasks.
      2. Explain that you will be unable to provide help and the reasoning behind it.
      3. Explain how to "think aloud" and the reasoning behind it.
      4. To demonstrate how the game works, give a short demo of the system.
         1. During the demo, show tool tips to the participant.
2. Task 1 [Easy]: Buying (creating) an account: Ask the participant to buy an account.
3. Task 2 [Medium]: Rearranging the position of an account: Ask the participant to move an account to a different position.
4. Task 3 [Difficult]: Moving money from one account to another: Ask the participant to transfer $X from one account to another.
5. Outro
   1. Ask the user which task he/she found the most difficult; which task was the easiest -- for both questions, ask the reasoning behind the participant's selection.
   2. What would he/she recommend improving (interface-wise) and why?
   3. Thank the user.

Test Measures

1. The amount of time it took for a participant to finish a task
2. The number of logged "positive" events (e.g., user says "cool" upon seeing something they like) for a task
3. The number of logged "negative" events (errors) for a task
4. The number of times a participant hesitated because he/she was unsure how to proceed (per task)