HeuristicEvaluation-Group:Group 4
From CS 160 Fall 2008
Heuristic Evaluation for Group:BuTtErFlY
Problem
It's an RPG game that helps kids who have experience with RPG (Role Play Games) learn math.
Violations Found
[H2-1:Visibility of system status]
1. [H2-1:Visibility of system status] [severity = 2]
There is no status (progress bar, etc) to show how far along the stage (level) the player has made.
2. [H2-1:Visibility of system status] [severity = 2]
There si no explicit goal for the game itself.
3. [H2-1:Visibility of system status] [severity = 0]
The number of stages in a typical RPG game is not presented.
4. [H2-1:Visibility of system status] [severity = 2]
All the windows/levels/stages should be labeled. [profile: 51G (what does G stand for? - the last box). + no label]
5. [H2-1:Visibility of system status] [severity = 2]
The player doesn't know how many HP the monster has left.
6. [H2-1:Visibility of system status] [severity = 2]
It is not clear whether the character loses or receives HP after finishing a math problem in the fighting mode because the number of HP change does not have a visible "+"/"-" sign or green/red color indication associated with it.
7. [H2-1:Visibility of system status] [severity = 2]
There's no indication of low HP(Health Point) low status for the player.
8. [H2-1:Visibility of system status] [severity = 1]
The dungeons icons and information signs do not stand out from other items in the world map.
[H2-2:Match system and real world]
[H2-3:User control and freedom]
9. [H2-3:User control and freedom] [severity = 2]
The player can't go back from crossing the bridge. The player is forced to solve the puzzle. If he/she can't solve the puzzle, then he/she is stuck there forever and can't go back to portal - can't quit game.
10. [H2-3:User control and freedom] [severity = 2]
During some battles, the "escape" button doesn't work - maybe it doesn't work intentionally.
[H2-4:Consistency and standards]
[H2-5:Error prevention]
11. [H2-5:Error prevention] [severity = 2]
The game doesn't prevent from accidentally exiting it.
12. [H2-5:Error prevention] [severity = 3]
The player was able to gain "subtraction skill" even though technically the skill was not learned (the player did all the math wrong).
13. [H2-5:Error prevention] [severity = 2]
when the mushroom is eaten, you can't eat it again but it doesn't disappear
[H2-6:Recognition rather than recall]
[H2-7:Flexibility and efficiency of use]
14. [H2-7:Flexibility and efficiency of use] [severity = 1]
The game is tailored to RPG gamers, but others will have to learn RPG game in order to learn math, which defies the purpose of the game.
[H2-8:Aesthetic and minimalist design]
15. [H2-8:Aesthetic and minimalist design] [severity = 2]
The math games themselves should be presented in a more "attractive" way, so the user will have more insentive and fun sloving math problems.
16. [H2-8:Aesthetic and minimalist design] [severity = 2]
The game should have a name.
17. [H2-8:Aesthetic and minimalist design] [severity = 1]
The main screen should be more relevant to what the game is about.
[H2-9:Help users recognize, diagnose and recover from errors]
18. [H2-9:Help users recognize, diagnose and recover from errors] [severity = 2]
After a question is answered incorrectly, the indication is unclear.
[H2-10:Help and documentation]
19. [H2-10:Help and documentation] [severity = 4]
There is no tutorials in the beginning for inexperienced game players to learn playing the game.
20. [H2-10:Help and documentation] [severity = 3]
Some functions (attack, skill, items) in battle mode were not clearly explained in the Readme document.
Summary of violations
There are a lot of visibility problems in the game. It provides very minimal status information for players so that players may lose their track of where they are. That means, if they take a break for the game they probably forget how far they have gone through in the game. In addition, the player is unable to go back to redo some tasks, such as fight a monster or cross a bridge, during the game. Moreover, when some actions are not permitted or unavailable, such as escaping the fight with moster, the disabled button is still visible and may confuse the player.
Aesthetic and minimalist design and Error prevention are the categories with the second most mistakes. The design of math problems' appearances and riddles is slightly overpowered by the game itself given that the whole point of the game is not the game but teaching players math.
Lastly, Help and documentation category has a couple of major or critical problems. The game does not provide tutorials in the beginning. The main screen of the game does not seem to be relevant to the game itself or its goal (teaching math), and it also lacks a relevant game name.
Summary Information
Violations found in each category
| Heuristic Categories | Violation Count |
|---|---|
| [H2-1:Visibility of system status]: | 8 |
| [H2-2:Match system and real world] : | 0 |
| [H2-3:User control and freedom] : | 2 |
| [H2-4:Consistency and standards]: | 0 |
| [H2-5:Error prevention]: | 3 |
| [H2-6:Recognition rather than recall]: | 0 |
| [H2-7:Flexibility and efficiency of use]: | 1 |
| [H2-8:Aesthetic and minimalist design]: | 3 |
| [H2-9:Help users recognize, diagnose and recover from errors]: | 1 |
| [H2-10:Help and documentation]: | 2 |
Violations in the entire interface
20
Violations in each severity category
| Severity | Count | |
|---|---|---|
| 0 -don’t agree that this is a usability problem | 0 | |
| 1 -cosmetic problem | 3 | |
| 2 -minor usability problem | 13 | |
| 3 -major usability problem; important to fix | 3 | |
| 4 -usability catastrophe; imperative to fix | 1 |
Recommendations
1. The main point of this game is to teach players math. So whenever the player encounters an opponent, he/she must answer given questions correctly to kill this opponent otherwise he/she will lose HP for wrong answers. It was possible to play the game and kill monsters without actually doing any math, and at the end of the quest a message appeared that the player can use subtraction magic, when he/she actually didn't do any subtractions at all.
2. There should be a map to show where the player is at in the world.
3. Each level's missions(objectives) should be clearly stated.
4. Each shown window in the game should be labeled. The game should show system status when you look at your profile. If there are no items, there should be a message saying "No Items". And in general, the player should know where he/she is (at what level/window/menu) at any point during the game.
5. The "Game Over" screen should have a "Replay" or "Main Menu" button.
6. If HP is low, should advise the player not to enter battle.
7. There is no constructively suggestion (ex: should suggest the users to look up Fib sequence if the players make too many misses.
- (a "X" or "-xyz" health points)
8. The game should always have a help button visible in case player doesn't know what to do.
9."Escape" button doesn't work - it should be mentioned - grayed out or a popped-up message. - confuses the player
10. The game should keep track of what types of mistakes the player makes (player stats) and then point out the weaknesses.
11. Is there a goal to this game? for example, after all the portals are conquered, what is the reward? how many stages are there? this would help to provide incentives to the players. there should be some sort of status message that lets them know how many portals they have conquer and how many left they still need to conquer.
12. We thought that there should be a message/indication that lets the users know there's a portal and information such as how to enter it, etc.
13. When solving a problem, there should be a limit to how many misses the users can make and 'hint' as to how to solve the problem should be more direct. For example, the hint should tell the user to look up and learn Fibinanci sequence instead of just saying "try to figure out the pattern." This makes the game more interesting and serves the academic purpose better because it prevent the users from just guessing numbers blindly and randomly.
14. We suggest that the sign posts for the portals should stand out. It'd be nicer if the posts are more attention-catching and labeled.
