PilotStudy-Group:一三三七-BudaChiou
From CS 160 Fall 2008
Contents |
Introduction (5 points)
The system we are evaluating is Fit Wars, which is an iPhone running game created for people who like playing video games and are trying to change their selves by doing exercise but without motivation. In FitWars, the user plays can choose either fight against the cpu or against another human through running. The player will earn points according to his/her effort after clearing each stage, and then he/she can either use these points to buy some in-game items or transfer the points to the games on our exclusive website. We will implement the game on the iPhone platform at the end, but we used Flash to implement this prototype for early stage testing. The purpose of this experiment is to get some quantitative data through the qualitative observations so that we can inspect the implicit flaw which we can’t find by ourselves, and then we can find out where to change to make the game interface better.
Implementation and Improvements(15 points)
1. Removing hover elements from the interface because we realized that although the hovering element can solve a lot of problems, since iPhone doesn't have cursor, the hover elements is actually useless on iPhone.
2. Implementing Game Settings which is an interface that allows users to configure the game’s settings. They can decide whether they want to turn the Game Music on or off, turn the announcer’s voice on or off, enable game sounds or not and finally decide whether they want to go for a more casual form of play, which means users can play through the game without doing action other than running although then they will get less points and don’t get the title of defeating another player. This feature was implemented based on the feedback we got from one of the users when experimenting with the Low-Fi prototype, which proposed the problem about feeling embarrassed if running in public.
3. Implementing Manage Stats, which is an interface that provides users with information about their character, their equipment, their stats, and their ranking in comparison to others. The ranking is portrayed by a graph that shows where they stand in comparison to other users. The rankings are dependent on the overall distance ran by users.
Method (10 points)
Participant
The participant I selected is a normal 50 year old household who does running regularly. I selected this participant because testers we selected are all Berkeley students, and since Berkeley students are usually smarter than normal people, the test result we get from them can not represent the majority. I conducted this testing at the user’s home because I want to make sure that the user is an environment that she feels most familiar with. The user used her desktop with windows platform to run through the interface, and I use a timer to record the log of her actions.
The three task I ask her to perform are:
1. Register and Log-in (Easy Task):
This is the first task users have to perform before they enter the main menu. I was wondering whether user will be confused by the go down button at the bottom of the register screen.
2. Send Message to other User (Medium Task):
User can send message to other user that is currently playing the game. I was wondering whether the user can find the mail box.
3. Play through the game (Hard Task):
The main part of the game, I was looking for users suggestion about this part.
I had the user sign the consent form and briefly explained what our project was about and also inform her what we have now and what we are going to have in the future. I then ask her to inform me whenever she felt confused or anything she felt was great. Then I ask her to start the first task (register and log-in), and at the same time I start the timer. After she finish each task, I would ask her to perform the next task and recorded the time she used to finish the task.
Test Measures (5 points)
The user actually got stuck very often, and that’s what I want. Although she always asked me “Is that right?” or “what should I do now?” I can’t tell her the answer because that would affect the result. However, every time she had question I would ask her more details about what made her get confused and what she expected to see, and then record the time and problem. After finishing all the tasks, I also asked her about the suggestion to these problems and things she feel is good about the interface.
Results (10 points)
Problem:
1. When I asked the user to register and log-in, she typed the username and password at the log-in page as I expected, but she get stuck at the character selection menu. The problem she had was that she didn’t know how to continue. Although she finally finished the task, she spent about five minutes on the task.
2. When I asked her to send the message to me, she didn’t know what to do at first, but then she clicked “you have 1 new message” button, and then she read through the message we initially put in the message box but still can’t figure out how to send the message. She spent about 30 second to find the “New Message” button and click it.
3. When the first time she sent the message, she sent the message to herself because she typed her user name next to “user name” on message sending menu.
4. She got confused about what to do after she defeated the first monster on the road.
Suggestion:
1. The main menu should be the first page that user would see after they enter the game instead of the log-in page.
2. There should a function for each page that allow user to go to main menu directly.
3. There should be a “pause” option during the running.
4. There should be a boss at the end of the stage.
5. This game is too easy.
Discussion (15 points)
Problem:
1. The user get stuck at the character selection page because she expected that she would enter the running mode right after she click the character. This expectation came from the usual fighting game, so we might make a change to allow user log into the main menu right after they select the character.
2. Although might intuitively click “you have 1 new message” to read message, it’s not an intuitive sense that it’s where we can write message, so we might change the option to “mail box (1 new message)” And the “New Message” button is not very obvious since it’s located right next to go back which doesn’t make intuitive sense, so we might make the buttons more like buttons.
3. When the user saw the question “user name” she intuitively though it was asking her user name, so instead of asking “user name”, we might change it to “target user” or just “to”, which make it more intuitive.
4. The user get stuck by the confirmation message after defeating the monster because she thought the “go back” button means go to the main menu”, so we might change that confirmation button to “continue”
Suggestion:
1. We might add some brief explanation about why user have to register and add an option on main menu that allow user to log-out so that the menu will return to the log-in page
2. We might add “back to main menu” option on each page
3. We may add an pause option for running part so that user can have option that save the game or go back to main menu
4. We might add an enemy that is more stronger than others at the end of each stage, and the user have to defeat it to pass
5. In real game the frequency and action will vary.
Appendices (5 points)
Materials (all things you read --- demo script, instructions -- or handed to the participant -- task instructions) Raw data (i.e., entire merged critical incident logs)
demo script:
This program was a prototype of an iphone running game, but it's designed in Flash to demonstrate the three important tasks that you are going to perform for our tests. As a result, a few of the functions are not working. The prototype also uses a number of fonts...these may or may not display correctly. If any text looks out of line, this is probably why. Walking is emulated by tapping left and right on the keyboard. In real life it would take much longer to complete a course, the tapping and progress you move has been sped up though for the sake of time saving. Space is used to symbolize an action being done. In the demo, the only action shown is jump. in the real game, the actions will vary both in time to react and the action required. Actions are measured by player motions.
critical incident log:
0: 00 start first task: Register and log-in
0: 12 click register
0: 23 click go down
0: 25 select an character
3: 14 click register
3: 40 click log-in
Total time for complete register and log-in: 3:40
3: 43 click "you have 1 new message"
3: 45 click massage name
4: 36 click go back
5: 08 click New Message
5: 32 click send
Total time for complete message sending: 1:52
5: 43 click fight computer
5: 44 click start new battle
6: 10 fight first minion
6: 44 finish fighting
7: 38 click go back
7: 55 fight another minion
8: 10 finish fighting
8: 11 click go back
8: 20 clear the stage
Total time for running: 2:48
