HeuristicEvaluation-Group:!Xobile

From CS 160 Fall 2008

Contents 1 TEAM 2 Problem 3 Violations Found 4 Summary of Violations 5 Recommendations

TEAM

• Mu-Qing Jing
• Geoffrey Lee
• Perry Lee
• Shaharyar Muzaffar
• Bing Wang

Problem

Tamagassi is a game designed to lessen gasoline usage by reflecting driving behavior on the state of a character that users choose. The game hopes to address the issue that users should drive at an optimal speed and acceleration to minimize gasoline usage. The game is location based mobile application that tracks user's driving information through GPS and possibly cell towers.

Violations Found

1. Cosmetic problem
2. Minor usability problem
3. Major usability problem: important to fix
4. Usability catastrophe: imperative to fix

problem# [heuristic violated][severity]

1. [H2-1:Visibility of System Status] [Severity = 2]: The confirmation dialogs are too generalized. For example, rather than displaying "Press OK to buy the top hat", the purchase confirmation dialog displays "Press OK to buy". I can see situations in which a user gets slightly sidetracked due to a conversation or other task, comes back to the game, and doesn't remember what he/she was buying, thereby being forced to click cancel, re-click the item, and click OK to confirm the purchase.
2. [H2-1:Visibility of System Status] [Severity = 2]: Along the same line as the previous bullet, upon purchasing an item, the item does not disappear and remains in the list of purchasable items. Is this intended behavior? Can you buy an item multiple times? The only feedback is a message that states "You have purchased this item". This may leave a user confused as to whether the operation was successful or not -- after all, the item is still in the list of purchasable items, and the message is once again too generalized: Which item was purchased?
3. [H2-1:Visibility of System Status] [Severity = 2]: No confirmation/indication that the phone is collecting data.
4. [H2-1:Visibility of system status] [Severity = 2]: Even though you can see the image of the item you currently have, there's no way to determine what it actually is unless you reference it to the names in the shopping screen.
5. [H2-1 Visibility of system status] [Severity = 3]: The user cannot see what accessories he/she has purchased from the Shopping menu. Instead, the user must tab back-and-forth between My Items and Shopping.
6. [H2-1:Visibility of system status] [Severity = 3]: When the user tries to equip or de-equip an item. The image shows what happens after the action occurs. There should be some status for before and after. Without the before and after, it seems confusing what the user is equipping / de-equipping.
7. [H2-1:Visibility of system status] [Severity = 3]: It is not clear how money is being earned, or when it occurs.
8. [H2-1:Visibility of System Status] [Severity = 3]: When shopping, after clicking an item, you are given a message which states if you wish to buy the item or cancel. However, there is no indication to which item you've selected. It would be useful to show which item was selected, so that the user can verify it is the item they want.
9. [H2-1:Visibility of System Status] [Severity = 3]: It appears as if there is no indication of how close a user is to achieving their goal. It appears as if they can only view data from one trip, but not how many miles they've travel so far since setting their goal.
10. [H2-1:Visibility of system status] [Severity = 3]: On the top of the main screen, it says Press "Ok" to begin collecting data. There's no system status once Ok is pressed.
11. [H2-2:Match system and real world] AND [H2-4:Consistency and standards] [Severity = 3]: Since this is going to be on a touch screen phone, the designer should think in terms of touch screens. Some of the buttons such as "Ok" to buy or "End" to cancel is very small. The buttons themselves are small. They should be expanded for the phone standard.
12. [H2-2:Match system and real world] [Severity = 4]: On the Char Home screen, I do not know how my driving habits influence the mood of my avatar. If my avatar does not feel well, is it because I am driving too fast, or because I am accelerating too quickly?
13. [H2-2:Match system and real world] [Severity = 4]: Equiping and de-equiping items should not be the same icon.
14. [H2-3:User Control and Freedom] [Severity = 2]: Although the prototype documentation states that "the [main] menu can be closed by clicking "Main Menu" again", this functionality was not working for me. Upon clicking "Main Menu", the six menus (e.g., review data) appear, blocking the "Main Menu" button. There is no way to close the menu without clicking on one of the six menus. As reference, I was using Firefox 3.0.3 in Mac OS X.
15. [H2-3:User Control and Freedom] [Severity = 2]: There isn't a way to pause collecting data for an unexpected event that the user does not want reflected in their data (eg emergency trip to the hospital).
16. [H2-3:User control and freedom] [Severity = 3]: The result only shows very little data regarding the driving behavior. Maybe you can implement a "for more details" button which will take them to a screen with more details.
17. [H2-3:User control and freedom] AND [H2-7:Flexibility and efficiency of use] [Severity 3]: On the reviewing data page. Once the user click on one data, the pop-up screen prevents user from going to other data points.
18. [H2-3:Control and freedom] [Severity = 4]: Goal cannot be changed once set.
19. [H2-3: Control and freedom] [Severity = 4]: The mileage goal set button seems to be linked to the carpool field.
20. [H2-3:User control and freedom] [Severity = 4]: When menu is pressed, it covers up half of the screen. The main menu button has to be pressed again in order for it to disappear. From the presentation, the group mentioned that this will initially be implemented on a touch screen phone. It makes more sense that the main menu would disappear after click any spot on the screen.
21. [H2-4: Consistency and Standards] [Severity = 1]: Although a confirmation dialog is displayed for purchasing and equipping items, no confirmation dialog is displayed when setting goals.
22. [H2-4:Consistency and standards] AND [H2-3:User control and freedom] [Serverity = 2]: When a user equips an item, the character is not reflect at all locations such as the main menu with the hat. Since the reward system is to equip the characters, it seems giving user a consistent view reduces confusion and give user a quick visual feedback of what they can get for driving well.
23. [H2-5:Error Prevention] [Severity = 2]: There does not appear to be anything that checks for errors (or at least displays error messages), specifically on the Set Goal screen for invalid inputs.
24. [H2-5/H2-9: Error Prevention/Help Users Recognize, Diagnose and Recover from Errors] [Severity = 2]: As far as I can tell, there is little done to help user recognize errors. No feedback is given when invalid input is entered into the "Set Goals" fields.
25. [H2-5 Error prevention] [Severity = 4]: In the shopping dialogue, the user can purchase an accessory item that he/she already owns, potentially wasting money.
26. [H2-5 Error prevention] [Severity = 4]: You cannot undo or sell a purchased item.
27. [H2-5 Error prevention] [Severity = 4]: You can begin collecting data before setting goals. There should be a forcing function to setup goals before collecting data.
28. [H2-6:Recognition Rather Than Recall] [Severity = 2]: It is not clear how you equip an item. When I go to the my items page, all I see is a list of items, and no obvious way to equip them. I had to test different things before I found the correct way to equip the item.
29. [H2-10:Help and Documentation] [Severity = 2]: If it wasn't for the blurb in the prototype documentation that states "Recent trouble spots will appear numbered" I would have no idea what the numbers refer to in the "Review Data" menu.
30. [H2-10:Help and Documentation] [Severity = 2]: There does not seem to be any sort of tips or help functionality in the application itself.

Summary of Violations

Violation Summary Table

Heuristic / Severity	1	2	3	4	Subtotal
H2-1	0	4	6	0	10
H2-2	0	0	1	2	3
H2-3	0	2	3	3	8
H2-4	1	1	0	0	2
H2-5	0	2	0	4	6
H2-6	0	1	0	0	1
H2-7	0	0	1	0	1
H2-8	0	0	0	0	0
H2-9	0	1	0	0	1
H2-10	0	2	0	0	2
Total	1	13	11	9	34

Based on the violations that we found, we formed a violation summary table that categorized violations based on the heuristics as well as the severity. The severity are pretty evenly spread out from 2 - 4 as we concentrated more heavily on the functionality asepct of the evaluation rather the cosmetic ones. The heuristics that received many severity = 4 were H2-2, H2-3 and H2-5. For H2-2 which titles "Match system and real world", the designers should remember that this system will be used on a mobile application, and thus develop the application according to the match of mobile systems rather than traditional workstations. For H2-3 which titles "User control and freedom", the designer should keep in mind what flexibilities should be granted while using the system. The designer should keep in mind what users want and perhaps how to give user more control of what they need. For example, it is imperative for user to reset goals but the prototype does not allow it. Lastly, H2-5 is about Error prevention. The designer should keep in mind how to help user prevent mistakes. This would help reduce frustration and make user more willing to play the game. For example, users could start collecting data before goals were set. Users should always be asked to set goals and proper feedback should be given.

In terms of violations, the heuristic that came out on top is H2-1:Visibility of system status. From the list of violations we came up with, the system lacks the feedback functionality where the user can be notified of system status. Without feedback and system status, users does not know what state they are in. Sometimes, it would generate frustration. When designing the system, the designer should keep in mind the concept regarding visual feedbacks, and these feedbacks should be provided whenever possible. The designer should treat themselves as first time users.

Recommendations

1. Test your prototype in multiple browsers, or more importantly, state which browsers are supported. At this point, I'm guessing it'd be too late to switch to a more platform-compatible framework like Flash.
2. Make your messages more specific; rather than "You purchased this item", state which item was purchased (e.g., "You purchased the top hat").
3. Along the same lines, you may want to give more visual feedback to operations. For example, I mentioned that when I purchased an item, the item remained in the list. If the item flashed or disappeared, it'd be more obvious that the purchase was successful -- having to check "My Items" to see if the purchase went through seems excessive.
4. Provide more help to the user, specifically in the "Review Data" screen. What do the numbers mean? I realize that screen space is limited, but if it is possible, a legend might be useful -- maybe when a user presses a button, a dialog appears that provides a quick legend with each number matched to the problem associated with it.
5. Add error feedback using the dialogs you've already implemented.
6. Since this will be used on the mobile interface. The designer should really keep the standards of mobile interfaces in mind. Keys that seem large on the computer screen might seem very small on the mobile screen. One of the example is the review data screen. With the icons so small, it would be difficult to press accurately on the icon.
7. When designing on mobile interface, the designer should keep in mind ways to maximize screen space while not blocking other features. For example, the main menu blocks half of the screen, and review data pop up blocks other potential data points on the map.
8. Not much user feedback is provided especially on the home screen when it says to press Ok to start collecting data. A user might wonder what is the status of the system. Perhaps the system can show the goals that they are trying to reach or how well they are doing so far this week.
9. The Store should disable already-purchased items, and the items should include some kind of "purchased" indicator.
10. There should be a way to undo item purchases, such as selling it.
11. The Char Home screen should provide driving tips based on the user's driving habits.
12. The Equip screen needs a separate de-equip button.
13. There needs to be a forcing function to setup goals before collecting data.
14. The Char Home screen should show when/why money is being earned.
15. There should be some way of letting user customize how many days of data they hope to collect with default being 7. 7 days seem to be a long time, the user might not even remember what he/she is doing. A more details button is necessary to display more details such as time of day and possibly previous driving habits at the same location.
16. It would be nice if there were several ways to do things, such as drag an item onto the character to equip items.
17. It could prove useful to not only set a weekly goal, but also a daily goal as well.