From CS 160 Fall 2008

Introduction

Introduce the system being evaluated

The system being evaluated is about how you can eat a balanced diet. It’s a fun way to learn more about your body organs and how much calories, fat, carbohydrates and protein your body can intake during a single day. The way the game works is as follows: In each wave/day of foods will enter the stomach; the player must balance his/her diet by either chosing to absorb or remove this type of food from the body. Clicking on the shift key will change the option to absorption, and the apple key or the windows key to change the option to remove/delete this type of food from my stomach. Once a food exits the stomach through the bottom of the screen, the intestines will absorb it. Hence its calorie, fat, carbohydrates, and protein amounts will be added to the total nutrition for that wave.

State the purpose and rationale of the experiment

The purpose and rationale of this experiment is to do a different kind of usability test so that we can gain more knowledge on how to improve our prototype. The purpose of this assignment is to give the user a usable prototype that will give them no constraints on performing the three different tasks, ranging in difficulty from easy to hard. Once the user can easily navigate through the game, we can now refurbish and reassess any implementations before testing the prototype with a more participants.

Implementation and Improvements

Implemented:

GamePlay

-Added the absorption mode and the delete/remove mode to absorb the selected food, or to remove the selected type of food, by clicking on the shift or the apple key to switch the mode.

Improved:

Highscore database

-Should list only top 10 scores, sorted by highest score first

Gameplay

-integrated upgraded organs with speed/clicks of food -integrated calorie bar chart with the physical characteristics (age, height, weight) that the user enters in the settings screen, but didn’t estimate the fat proteins and carbs values yet.

-changed the default nutrients values from 10/10/10 to fat:60g carb:300g Protein:50g.

- Health doesn’t get reset after each wave, but instead we now have a reset health button in the upgrade option.

-Clicking foods now gives more points, to balance the game play, so that the user can upgrade organs.

- The score used to get subtracted whenever you spend points to upgrade an organ. But now we have two separate variables for scores - one to keep track of your total score, and one to keep track of you upgrade points.

Upgrading prgans

-added in-game info about organs (what the organ does, how many points to upgrade each level)

Settings

-allow users to enter height, weight, age, etc. -coordinate gameplay with info that the player enters

Method

Participant

The chosen participant had no experience playing similar games, but is a tach savvy and was interested in becoming healthier. The reason for choosing this participant is due to the following reasons: the participant will have very minor technological issues. Also so that I can evaluate how user-friendly is the interface for a person who might see it for the first time (ie: was it appealing to the participant or not?!). Finding the target participant is done by randomly asking people around moffit library (a good pool of people who are willing to take a break from studying) and asking couple preliminary questions to eliminate non-target participant. If s/he fit the type of user I was looking for, I would then schedule the prototype testing with that participant.

Apparatus

Equipments were: 1) Laptop, for ease of access and to use in the external room. 2) External mouse: since the game is better played by an external mouse, since some users might not feel comfortable playing or having control with the laptop pad. 3) Timer: to time the user’s response times.

This was done in a quiet room to allow the user concentrate on the game and let them speak out load of what they are thinking.

Tasks (1/2 page)

Playing the Game (Hard difficulty)

This task involves performing the main goal of our game: the gameplay (i.e. defend the body against eating too much and over shooting your target calorie, fat, carbs and proteins count/intake ). This is done in the main gameplay screen, in which waves of foods will enter the stomach; the player must selectively choose or remove the pieces of foods to pass through the stomach (a way of absorption), absorb or remove a piece of food (by the new mouse functionality, shift and apple key respectively). Once a food exits the stomach (through the bottom of the screen), it will be absorbed by the intestines and its calorie, fat, carbohydrates, and protein amounts will be added to the total nutrition for that wave. If too much or too little food passes through the stomach, the health bar will go down (this is not implemented correctly yet). once it hits zero, the game has ended. Since each wave represents a day, the goal of each wave is to allow the desired daily values of the respective nutrients into the body. Additionally, the stomach, liver, and heart can be upgraded to aid in gameplay, or reset the health bar. (upgrading these organs is our next task).

While testing: I looked to see how fast will the user decide which foods to eliminate after they hover over the food object, and if they understood how to play the game correctly.

Upgrading Organs (Moderate difficulty)

At the end of each wave, the gameplay screen will switch to the upgrade screen. This screen contains the pictures of three organs—heart, liver, and stomach—that can be upgraded by allocating points (acquired during gameply) to those organs. Also the user is allowed to use the points to reset the health bar, to full strength. When the mouse rolls over the picture of each organ, text in the upper right of the screen will display information about the organ, its effects in the game, and how many points are needed to upgrade the organ to each level. Upgrading the organ is categorized as a moderate task because the player can't just upgrade whichever organ that is shown on the screen; each organ has a different, positive effect on gameplay. The player has to thoroughly think about which organ to upgrade, as the player can't undo the upgrade once it is done. Once the user is done upgrading the organs, he or she can choose to either start the next wave or go to the menu.

While testing: I looked to see how fast would the user reach the goal of upgrade after s/he reaches the upgrade screen. Will s/he understand that once they click on upgrade, that s/he have upgraded that organ.

Submitting a High Score (Easy difficulty)

Submitting score should be easy and pretty straightforward. There are two conditions that have to be satisfied for a player to submit his/her score: either the player quits in the middle of the game or the player loses the game. If the player wants to quit in the middle of the game, the player must click the menu button (available in the gameplay and upgrade screens) and choose the appropriate option to submit his or her score. In addition, if the player dies (the health bar goes to zero), the player will automatically be given an option to submit his or her highscore. The actual process of submitting the score is done by typing a nickname or initials and clicking submit button. While testing: I looked to see if the user was confused on how to submit the high score. And how fast will they get to the high score screen.

Procedure

I started the pilot study first by testing the system on my own and see if all the tasks were implemented correctly (this was kind of the pre-pilot study). As for the pilot study itself, I started by giving the participant a short demo of the system, by performing an unrelated task (ie: it was not one of the three tasks being tested). It was also an introduction of how the system works in general. When I met with the test user, I greeted him warmly so that he would feel comfortable in the testing environment. Then I introduce the background behind the test and how his input is critical to encourage him to speak out load of what he was thinking. As well as explain the themes and goals of the cs160 class, and also elaborate on the purpose of the test, all the while asserting its confidentiality. Finally, I presented the user with one task at a time. Tasks that I wanted them to accomplish, making sure that he understood that any difficulty with achieving the tasks was not their fault, but rather a fault of the design. Once the test was in session, I encouraged the user to express his or her thoughts, by asking questions such as, “What are you thinking right now?”, while he was waiting for more foods to come down, or if I saw a confusion look on his face. I was also measuring how fast does the user respond to certain events and how long did it take him to figure out how to delete or absorb a food object. I started first by measuring each task at a time. I also took notes, as a camera was not available. I explained to the user that I was documenting their input and suggestions. I didn’t want the user to feel under pressure that their performance was being measured. Finally after I finished the pilot study with the user, I sat down and looked over the critical incidents and how it affects out design.

Test Measures

I have measured the following dependent variables:
1) In Game Task time:
Reason: This is because the game should not be boring and should engage the player to continue playing the game while flowing through the logic correctly.
2) In the game play, how fast will they decide to click on an object, once the object appears Reason: This was tested to see if the user understood how does the concept of the game really works, also to see if it was obvious to click or just hover over the objects.
I also had a log of the following errors:
1) The user was very confused on how to play the game. Was it clicking on the object or was it hovering over it.
2) The user also upgraded every single organ, but though that he didn’t upgrade any, the feedback from subtracting the score was not enough for him.
3)Also after the user has died in the game, the message popped up “you need to eat more healthy food” He complained a lot about that one, since it didn’t give him more explanation of what is healthy, unhealthy or balanced.

Results (10 points)

	Time to Finish Task	# Positive Events	# Negative Events	# Times Hesitated
Task 1 (Play game)	~2 Min.	2	2	2
Task 2 (Upgrade organs)	~30 Sec.	0	1	0
Task 3 (Submit high score)	< 10 Sec.	0	0	0

Summary of the Process Data

Task 1 took the longest to accomplish because the participant had trouble understanding the instruction, and after getting the game, he continued playing the game, skipping the upgrade option.
"What should I do? Do I use the mouse or the keyboard?" [Hesitation]
"How should I use the mouse? Do I hover over the object or do I click, left or right?” [Hesitation]

 He assumed a hint would come up.

"I don’t see whats the point of having too many coconuts" [Negative]
"Need more healthy food to balance the diet, I can’t reach my goal”. [Negative]
"Good way of keeping me ingaged in the game" [Positive]
"I don’t know if I upgraded correctly? How does it work" [Negative]
He liked the information, but didn’t really read them, he was looking to read less and rely on hints.

General Feedback

He needed more food options/choices
He noticed the bars and thought they were a good way to project the goal.
Wanted to see a line of breakfast, lunch and dinner choices and then he can choose, he felt the food choices were unrealistic.
The you need more healthy food when the user dies, was in adequate explanation of the end of the game.

Discussion

This pilot run helped to reveal the ambiguity of the in-game mechanics (e.g., how do I make sure I upgraded?). The study have shown the problems that couldn’t have been shown in previous usability tests, for example the visual clues that were absent. The results were not all positive, however -- this pilot study also showed us that we still have some in-game problems to discuss and make sure they abide with the educational theme. One upgrade-game mechanic that needs to be addressed is feedback – When a player upgrades, there aren’t enough clues to let the user know if he or she has upgraded successfully. Also in the ingame window the user might just start the game without looking at the instructions, so some hints that will allow the user to know what to do might be very helpful.

On the educational side, we need to better explain the why some food choices might have a worse side effect on the body, but in a better way than just in writing, I don’t know yet how I would integrate this into the game, but it seems to me that the user doesn’t want to read more information while trying to continue playing the game.

Also the user didn’t really get the concept of how the bar changes colors (or increase with a different color) when they hover over an object. So it seems to me as one of our number one priority now is to implement the tutorial and to put more clues into the game. Make it more realistic in terms of food choices and make sure our messages communicate well with the user

Appendices

Demo Script:

This experiment is for our CS160 User Interface class. The theme for this semester is to design a serious game, in which we go through the phases of brainstorming, selecting and interviewing a target user group, creating and testing a prototype, and finally building the entire product. We are currently in the phase of testing our medium-fidelity, or flash prototype, and you have been selected as one the testers (whose identities will not be revealed). Based on your feedback, we will be able to make changes and improve upon our prototype to make our design more appealing. Before you start this test, we want you to know that any difficulties or inconveniences you have with our prototype is not your fault, but the fault of our design. Please feel free to voice your questions, thoughts, and suggestions while you are testing our prototype. Our interface is almost completely implemented, so you will have access to pretty much all the functionalities of the game. The goal of our design is to see if our users can easily perform specific tasks. Thus, we will present you with three separate tasks that we want you to complete. I will present the tasks to you one at a time, and I want you to show me how to complete them by using our interface. Don’t feel bad if you are having trouble completing a task-that would be our fault for creating a poor design. Any questions?

Raw data

1-Confused playing the game
2- trying to balance diet
3-too much coconuts
4-upgraded everything
5- the point system needs revision
6- thaought about breakfast, lunch dinner, for one day and not all the foods coming in
7- when upgrading should tell you that something has been upgraded on the side
8- “you need more healthy food” not a clear message when you die, it doesn’t explain why you died.
9- Submitting high score was very easy, and viewing it was intuitive.

PilotStudy-Group:Lucky Seven-Michael Boulos