Heuristic Evaluation
From CS 160 Fall 2008
Lecture on 9/24/2008
Readings
- Heuristic Evaluation. Nielsen. (Read the 5 linked pages under the heading: Jakob Nielsen's Online Writings on Heuristic Evaluation).
Discussion Questions
Nielsen's usability heuristics were developed by studying usability problems in a broad set of applications. He notes that there may be common problems with particular classes of applications that are not in the list. Think about how this might apply to games.
Discussions
Please post your critiques/commments on the required readings below. To do that, first login by using your user name and password, then click the "edit" tab on the top part of this page (between the "discussion" page and the "history" page), New to wikis? Read the Wiki editing guide. . Hint - Please put a whole line == ~~~~ == (literally) at the beginning of your submitted critique, so the wiki system will index, sign and date your submission automatically.
Perry Lee 19:27, 23 September 2008 (UTC)
Nielson's usability heuristics made a lot of sense to me. It seems like a natural accompaniment to user testing -- each has its own strengths and may catch usability problems that the other doesn't. In the case of heuristic evaluation, it provides a more focused approach by providing evaluators with a list of heuristics (e.g., visibility of system status) to use. This approach works well in a first-pass, quickly identifying "obvious" usability problems. User testing works well in a second-pass to identify the remaining non-obvious problems.
Several good (and somewhat obvious) points that I thought he made are:
- with the increase in the number of people (evaluators) comes an increase an in the number of usability problems found; no one individual can be expected to find all the usability problems
- it is important to isolate evaluators prior to aggregating all of the feedback -- I can certainly see situations in which one individual, or several individuals, can heavily bias the group's overall opinion
Mohammed Ali 20:06, 23 September 2008 (UTC)
Although there might be common heuristics between all kinds of application, game usability is definitely in another ballpark of user testing and evaluation. Game usability as compared to other software usability has some similarities and some differences.
A software menu, for example, and a game menu do share the same heuristics in ares such as visibility of system status, user control and freedom, consistency and standards, error prevention, recognition, and aesthetic and minimalist design. System status, player status, and other feedback is also similar in principle that the user should have all the relevant information displayed in a intuitive manner, critical information should stand out, and minimalist information design should be practiced.
However, game play, the core of the application can have a whole new set of heuristics. An example of a major heuristic that should fall under the class of games could be control mappings. This is vital in usability for gamers. Control should have a simple mapping and if it is complicated, players should only have to know this once or should be a part of the industry standards for games. Another area of game usability would be in level design where design should not make it easy for players to get stuck or lost and should have some sense of indicator of progress.
Jordan Berk 20:53, 23 September 2008 (UTC)
Nielsen's explanation of how to conduct a heuristic evaluation made a lot of sense to me. One thing that stood out was his mention of the possibility of conducting a "debriefing session" following the heuristic evaluation. Because the experimenter can't have much interaction with the evaluator as to guarantee unmuddled usability problem results, the debriefing session gives the evaluator to give his/her take on the usability features they encountered and give the observers/designers a time to ask questions of the evaluators experience that may help them on the next iteration of the design. Likewise, the evaluators themselves could provide input into what they would like to see to make their usability experience better. Also, as Nielsen said, it's really the only opportunity for the evaluator to provide input as to what did work for them and the positive features of the design.
Vedran Pogacnik 01:34, 24 September 2008 (UTC)
Some of Nielsen’s usability heuristics can be applied to games. However, some could not. Since we are not talking about “serious games” here, heuristics like documentation, or helping users recognize, diagnose, and recover from errors are completely pointless. Can you imagine playing Command and Conquer while struggling with the user’s manual? Or playing WoW and “recovering from errors” (whatever that would mean in practice)? I didn’t think so.
The thing is, for most software applications, the heuristics are sound, and conceptually, one should follow them in evaluating the usability of the project. But games are tricky, because most of the target users in games are small kids, with nothing better to do than play games. As such, they might have different ideas about usability, and in the case of children, again, some of the heuristics might be impossible to follow. For example, a child might not find a game to be flexible and efficient to use; maybe that is the whole point of the game- to make HIM adapt to the game; maybe making things obscure is the reason for playing in the first place. Secondly, some games (like Neverwinter Nights), have the basis that is so complex and sophisticated in their choice of menus, actions, puzzles, etc., that even though all of that does give user control and freedom per se, it makes error prevention, aesthetic minimalist design, efficiency of use, or a match between systems and the real world- so much more obscure.
Clearly, sometimes there is a tradeoff between heuristics. When talking about games, one must take into account the target audience and their wants and capabilities. As a template for generic applications, Nielsen’s heuristics work fine. With games- not so much.
Wenda Zhao 02:39, 24 September 2008 (UTC)
I have learned a lot from Nielsen's method to find problems in a user interface. I think that games require a lot of interaction from the user. Usability are extremely important for designer to pay extra attention to. A game will be totally ruined if it has plenty of usability issues. So designers should first identify common classes of usability problems seen in the kind of game that they are developing. And then categories them into smaller sets and come up with the solutions that avoid these usability problems.
Jonathan Fong 02:49, 24 September 2008 (UTC)
Because the motivation of a user of a game is usually entertainment, while a user of a business suite or other similar application is task-oriented, different heuristics apply.
Above all other priorities, games have to be fun (i.e. provide entertainment value). Abiding by the strict heuristics is a very stuffy and restrictive criteria for games. For example, repetition/reproducibility is valued for other applications, and this is checked by Nielsen's "consistency and standards" heuristic. Games, however, need some spontaneity and randomness. The user/player values being challenged and engaged. Specifically, Nielsen would frown on using (somewhat) random sound effects as feedback for the same event/action -- in a game, though, a player would dread hearing the same sound effects over and over. In fact, a game heuristic should be made to penalize for interfaces/interaction that requires/use repetition; the game would be just plain boring.
Gary Wu 04:36, 24 September 2008 (UTC)
When applied to games, most of the heuristics do make sense. However, some of the points, such as recognition rather than recall, consistency and standards, and matching the system with the real world do not apply. Since the game itself, is itself it's own world, a lot of the jargon used will not coincide with the real world; the whole point of a game is to escape the reality of the world. Also, if the user is not a familiar gamer, heuristic evaluation will not be a sufficient choice. As mentioned in the usability problems article, a proper evaluation will combine both user and heuristic evaluations. Games, in general, are tricky puzzles that almost always forces the user to adapt and go against the "standard." While it is possible to have a heuristically sound game, it would definitely be a very boring and tasteless one.
Haosi Chen 05:49, 24 September 2008 (UTC)
I really agree with Nielson's usability heuristics theory. And I think some of the points that he mentioned in the article are really good. First, the more evaluators you have the more number of usability problems you will find. Another point is that no one individual can be expected to find all the usability problems. These two points can be greatly apply to game designing.
Mike Kendall 05:59, 24 September 2008 (UTC)
When applying the heuristic list to games, most rules are a good idea. “Minimalism” is a borderline rule, since if you look at a screenshot from World of Warcraft, you'll see anything but minimalism, and it works.
The biggest offenders are the first two rules. First, the system status, in a lot of games, is meant to be hidden. When playing the 9 puzzle or the rubicks cube, the easiest way to get to the goal state is to take the game apart and put it back together in a winning configuration, but that's not allowed in the game. Although the “status” is clear, you can't actually know what's going on unless you have spent a considerable amount of time learning the games and the algorithms for solving them.
Secondly, (and I'm going to sound like a broken record) games are an escape from reality, so it's counterintuitive to make a game that resembles real life. Although something like The Sims does resemble real life, it is a simplification of real life such that your actions are governed by understandable rules... It's closer to Digimon than real life, so I think I can safely say that The Sims is still a counterexample for the second heuristic.
Buda Chiou 06:15, 24 September 2008 (UTC)
The heuristic evaluation seems a good way to get valuable feedback from the evaluators. However, it seems that the evaluators of the heuristic evaluation must be intelligent and patient so that they can give observers valuable feedbacks. Moreover, the evaluators must really care about the tested user interfaces which they probably won't use anymore. I guess the evaluators must be some experts who get benifits from the observers so that they would take the evaluation so seriously.
Bing Wang 06:48, 24 September 2008 (UTC)
The reading is somewhat interesting. Before I read this, I never thought about heuristic evaluations. I thought you would directly bring the product to the user testing phase. After reading the artcile, I realized the differences between heuristic evalutions and user testing. What I am still a little confused about is whether you should use experts in heuristic evaluations or should you just use regular users in the evaluations. Nathan talked about how in heuristic evaluations, the participants can ask questions and get answers from it while in the user testing phase, that will not be possible. It seems to me that if you have an expert do the heuristic evalutions, his/her question will be much more focused. You can then use the user to do user testing to see if the general public or your target audience know how to use your user interface efficiently. One of the things he mentioned was not to let the evaluators communicate with each other beforehand. This is definitely a must. Under the peer influence, people might start wondering if the error they found is valid or if the other person's error is better etc. Eliminate the participants interaction with each other beforehand can certainly reduce peer influence.
Kumar Garapaty 07:16, 24 September 2008 (UTC)
I can definitely see that Heuristic Evaluation can easily lead to a cost-benefit ratio of 48. what if the users were more involved in the design process more than through the Heuristic Evaluation? Would this be beneficial because the user can get what he or she wants or will the design become skewed because the customer doesn't exactly have a clue on what he wants? The severity ratings seem to run counter with the Task Analysis and Contextual Inquiry strategy. Asking the evaluator to give severity ratings after they used and thought about it, could lead them to forget or skew important information about the usability of the design. There are definitely common usability problems when discussing game applications beyond those/more specific than those listed in the article. Its mostly the logical flow of the game as well as the graphics that are main issues of usability.
Jeffrey Rosen 07:41, 24 September 2008 (UTC)
It is important to have concrete heuristics for UI. It helps move it from a hand wavy art into more of an engineering problem. It is interesting that in spite of this, the evaluators still have a fair variance in their reports. A game could benefit from this evaluation the same way other applications can. Each aspect could be judged based on its difficulty. At the end of the day, you would get a report saying how hard each evaluator judged various tasks, like starting a new game, or loading a save. However, what would be more difficult to judge, and more important, would be judging the actual difficulty of the game. For example, one user may find it very difficult to beat level one, but another person may find it easy. I would expect to see a video game having much greater variance than a media player application, for example. It is much harder to judge objectively.
Cynthia T. Hsu 07:45, 24 September 2008 (UTC)
Jimmy Nguyen 07:48, 24 September 2008 (UTC)
From the first article, I found it interesting in Figure 3 that shows that ratio of benefits to cost "peaks" when there are 3-4 evaluators. I feel this is because if there are too many opinions, as people try to improve the product, they use more resources to do as much as they can, but they end up doing too much and the overall gain in profit and benefit is not worth it. To me, I would still disagree because in my mind, I feel the more opinions the better and then just to base any decisions off of pure judgement.
The way that some common problems presented as heuristics might apply to games is complexity vs. simplicity. Even referring to what I was thinking before, sometimes too much results in a poor product. Another issue would be user control and freedom (in addition to help and documentation), which would be very important in context with this semester's theme of serious games. Overall, a user to a game (like the ones we are designing in class) will have to CLEARLY know what to do. If things are unclear from the start (and the interface is poor), the user could potentially give up before starting.
KevinFriedheim 07:50, 24 September 2008 (UTC)
For some games, in my experience, it takes a gamer to play them. That is to say, it takes someone who has had extensive experience playing games to be able to "innately" figure out how certian games work (without reading the directions first). This is because a gamer has had extensive experience with different gaming heuristics and usabliity features. However, unlike common applications like word proccessors or spreadsheets, games are much more diverse in that they come with a superiourly unique interface; each game is different in its own subtle way. This being said, one might argue that any "general" set of heuristic principles that would be needed for gaming design would be too broad to be useful.
Frank Yang 08:26, 24 September 2008 (UTC)
I think that games should follow some sort of heuristic, but at the same time, it would be extremely hard for such a heuristic to be made. There are simply too many different types of games out there. Any heuristic that would be created would be extremely broad, and therefore, not very useful. However, with most complex games nowadays, it is common for the first few minutes to be an introduction or tutorial to the game to allow for the player to be accustomed to the controls. But because games itself are an abstraction from everyday life, there is no way we can apply some of the heuristics mentioned in the readings.
Kai Lin Huang 08:38, 24 September 2008 (UTC)
Ease of playing and excitement are essential for games. Ease of playing gets users started to enjoy the game as soon as possible. However, games usually do not adapt the platform convection in interface designing. Since the objectives of most games are to have fun, users would easily give up playing if the controls, rules and settings are complicated and nothing would guide new players through the beginning of the game. Even though some players like challenges, a game with hints that tell users what to do and how to play it is more enjoyable for general players. Easy to learn actually apply to all applications except some very specialized software such as programming. Excitement is another common problem with games but not other application. Users generally will not feel excitement when using a spreadsheet application, but playing a game does. A game needs to generate some excitements to retain players to keep playing.
Alan McCreary 08:42, 24 September 2008 (UTC)
I like the idea of temporarily isolating the evaluators, not allowing them to speak with each other until the evaluations have been completed. If they were allowed to communicate during the evaluation process, the group of evaluators might underreport the number of interface flaws, or they could report a flaw as being far more severe than it really is (there's some psychological effect in which people will develop a stronger opinion on something after conferring with people who agree with them).
Coming up with a heuristic for judging the interface of a game doesn't seem too difficult, but making a heuristic for the actual gameplay - which is just as important, if not more important - would be hard. In a time frame of just an hour or two, it is difficult to predict how fun the game will be in the long run, and whether the gameplay is balanced (in a multiplayer game like Starcraft, for example).
Hao Luo 08:46, 24 September 2008 (UTC)
I think the usability heuristics theory provide many good points and can be applicable towards gaming, even though some points may not apply as well. With gaming, you have a narrower audience, so you don't need as many evaluators, but having many evaluators is still important in the sense that one person cannot find every problem and cannot represent the entire user base. This is especially important for games which appeal on several level, to casuals and hardcores. You need to have a wide variety of evaluators and using heuristics to find design flaws.
Paul Im 08:51, 24 September 2008 (UTC)
An additional usability heuristic that would apply only to games would be the amount of control a user has over the game. If the user is controlling a virtual character, the fluidity of the character’s movements would most likely disrupt the flow of the game. If playing a serious board game, the ability to precisely control your next move is a huge factor to the success of the game. Thus, users should have a reasonable amount of control over the game (as a side note: this is a different type of control than the one that Nielson mentions as a heuristic).
Of course, some of the other usability heuristics that Nielson provides are also applicable to games. Errors should be prevented or easy to diagnose, and games should be playable by both inexperienced and experienced gamers.
Witton Chou 09:09, 24 September 2008 (UTC)
I think it is important to use evaluators of the target user set. However, consideration could be put towards using evaluators outside of that set as well. There are cases where it is not possible to do so due to the focus of the product (possibly due to technical focus), but there are times when we should consider testing outside of the expected target users. A person who is outside of the target user set may be new to the environment type more sensitive to ease of operability issues or even more pessimistic and exaggerate minor flaws that may be worth looking at carefully.
I did find it quite interesting that three to five evaluators is the optimal number of evaluators. I was expecting something like at least 10 or 20 evaluators would be needed to effectively determine the flaws of user interface design. Although through figures 2 and 3, I do see how three to five evaluators is a reasonable choihce as problem discovery levels off at around five and cost effectiveness peaks at around four.
JoshuaKwan 09:14, 24 September 2008 (UTC)
I am a little upset with the idea of heuristic testing because in my opinion, every usability bug is one that must be fixed. The point of heuristic testing is to figure out which usability bugs are seen most often in order to prioritize them. I suppose companies save money by not fixing the ones that no one sees, but as long as the fix doesn't make other things less usable, there is no reason why such bugs shouldn't be fixed.
That said, the heuristics themselves are appropriate and represent principles that should be followed by every developer. Ideally this sort of thing shouldn't be necessary and the only usability bugs will be ones that are not obvious at all from the perspective of the developers.
Volodymyr Kalish 09:14, 24 September 2008 (UTC)
Jakob Nielsen's list of ten recommended heuristics for usable interface design, is a good guide for serious game interface design. However, this list is too general since it could be applied to any interface design. So in a specific case of serious games interface design, this list can be much broader and more focused. Also, for a new serious game it would be hard to find a good set of the evaluators that would accurately represent the target user group due to budget limitation. And the set of observers has to be really knowledgeable about the game under evaluation. A better idea would be a feedback from target user group that play the game. However, as the previous reading stated that if a product is “marked” bad due to bad interface, the users tend to avoid it, even though its interface was improved. But once again, since most of the serious games are freely available, this shouldn't be an issue.
Xuexin Zhang 09:35, 24 September 2008 (UTC)
Personally, I believe that Nielsen’s usability heuristics evaluation could be applied to games in general. However, games, as a special type of applications, require very high usability in order to allow players to engage in the gaming experience and have an extreme requirement in User Interface. Due to the nature of heuristics evaluation, which is a discount usability engineering, it could be possible that the heuristics evaluation unable to provide a comprehensive feedback of the system. Therefore, I believe that additional testing by greater number of users is necessary in order to test usability for games.
Karen Tran 09:52, 24 September 2008 (UTC)
I think the heuristics evaluation is a very helpful approach to designing user interface for games. It logically isolates different component of the process into individual problem, that could be treated and improved independently: “Heuristic evaluation is performed by having each individual evaluator inspect the interface alone. Only after all evaluations have been completed are the evaluators allowed to communicate and have their findings aggregated. This procedure is important in order to ensure independent and unbiased evaluations from each evaluator.” I also liked the 10 Usability Heuristics listed in the reading. I think they are all essential to programming UI for games. Visibility, user control and freedom, consistency and standards, flexibility and efficiency of use, and match between system and the real world are to me the most important evaluation features that a designer should take into considerations when designing games. I think while the idea of heuristic evaluation is really great, it’s too ideal to be implemented realistically. It is too difficult to address and break down the problem into independent smaller problems. Each problem might be influenced or influence the next and to completely isolate them seem too unrealistic of a problem to be done.
Kevin Lam 09:55, 24 September 2008 (UTC)
Of the ten usability heuristics, only one really stands out to me: recognition rather than recall. The other heuristics are trivial and are more often than not expected in most modern games (i.e. you would expect a game to display words in your language if the game has been promoted in your geographic region; furthermore, most players would deplore a game if an error message appeared with unreadable text). "Recognition rather than recall" brings to mind the idea that users should be focused on completing a task or meeting their objective(s) rather than trying to remember various pieces of the program. Unless we are referring to a memory game in which users are asked to memorize words or figures on a screen, a system should not require the user to "remember information from one part of the dialogue to another."
Trinhvo 09:59, 24 September 2008 (UTC)
Evaluators are to find usability problems. There must be at least 3 evaluators, but 5 evaluators at most. During evaluation session, evaluators work independently and exchange their evaluations when done. Nielsen's usability heuristics is beneficial for finding major problems, but it can't replace user testing. Both must be used. User testing will bring up problems the heuristic evaluation can't due to evaluators' overlook of usability problems. The problem is user testing can't be done for a short period. For example, Creed's Assassin is one the most successful games of Ubisoft, but there are many customers complaining about "it takes so many steps just to quit the game.
Geoffrey Lee 10:27, 24 September 2008 (UTC)
Nielson mentions alternating between heuristic evaluation and user testing. I think it would also be useful to first run through a comprehensive checklist of frequent usability mistakes after each change. Here's a quick (but incomplete) list that I came up with:
- If the feature is color coded, is there an alternative for color blind users?
- If the feature involves destroying or altering data, is there an undo option?
- Is the feature accessible by keyboard?
- Does the feature provide feedback to confirm an action?
When used after each feature is completed or altered, this type of concrete "yes or no" checklist acts sort of like a "paper heuristic evaluator". It is also even cheaper and less time-consuming than heuristic evaluation, but it is not a replacement, of course. So rather than alternating back-and-forth between heuristic evaluation and user testing, you're actually using increasingly more-expensive but more-effective usability testing techniques each time.
Jacekmw 12:23, 24 September 2008 (UTC)
Heuristic testing as Nielson describes it is a very useful tool in any project. As a rule, the allowance of sorting priorities of user interface bug fixes allows a huge project to be micromanaged and allows the group working on it to be able to allocate their debugging according to what is most important in the eyes of both the user and the heuristics. Heuristics, in this case, being excellent for the more common issues that we do can expect users to complain about, but user testing nonetheless still very necessary. I also found Nielson's idea of a user-designer 'debriefing' interesting as it reminded me of one of our first readings, which described the interview process as well as other ways to interact with users and see where they have issues. This process benefits the design of any application with regard to user interface, but is particularly significant with respect to games (serious or not) because there are often issues with gameplay (too difficult or too easy) that are just as much user interface problems, yet for the most part cannot be anticipated without user feedback.
Antony Setiawan 14:26, 24 September 2008 (UTC)
I found that nielsen's evaluation method to be suprisingly works well towards games. For example, in one of his principle, he mentions about the needs of emergency exit, undo, and redo as a part of user control and freedom. FIrst, I thought why should this elemenet exist in a game. The needs of emergency exit is understandable but why undo and redo? when I thought about it, the save and load feature can somehow be viewed as undo and redo. A strategy games that doesn't have undo/redo features would be very hard to be played. Minimalist design are certainly a very important elements nowadays programs / games needs to have, balanced with system status. Another problems that a lot of games made besides from nielsen's list is accessibility. For a game that completely uses mouse to control things in the games, some people who aren't comfortable or do not have mouse would not be able to play the game. A more accessible games that provides choice of control device are more seemengly better.
Yuta Morimoto 15:10, 24 September 2008 (UTC)
I think the heuristics evaluation is a helpful to designing user interface for games and important to improve the process for User Interface. Furthermore, it is strike me that individual evaluator performs only 35 percent of the usability problems in the interfaces. If it is true, to testing usability we need so diverse testers that we can set clear the problem. In the case of child testers, children may not find a game to be flexible and efficient to use. Since, they might have different ideas about usability, so they would make a strange mistake that can not be revealed in the case of older testers. As a result, we can summarize their behavior and maybe improve user interface.
Saliem Than 15:37, 24 September 2008 (UTC)
I thought Neilson's heuristics such as visibility,match and user control and visibility and such as good guiding principles that can be applied with game design in mind. This is even in light of the case that the problems encountered are different. I think that is the case for any sort of design, his principles are not an absolute in the design of tool like applications. With game designers like Shigeru Miyamoto leading the field, a more specific set of game application heuristics plays out. These sets appear to often revolve around three main categories of the game interface, the game mechanics and the game play. I am very wary of of such heuristics however, especially since as this article notes, the game interface is always changing, such as in pervasive games in smart home environments. Despite this, I do agree with some of the generalized, and specific rules of thumb including:
- That the player gets involved quickly and easily
- That AI be reasonable but unpredictable
- That there is a great storyline
- That control options are mapped and minimized and
- That sound is used to provide feedback
References:
Stuart Bottom 16:04, 24 September 2008 (UTC)
It is important to note that games (and other more "specialized" applications not addressed in depth by Neilsen) may actually, whether by definition, convention, or design, go against some of Neilsen's top ten heuristics. For example, games often DON'T "match up with the real world," on purpose. Reducing the "consistency and standards" may make a game more challenging for users, and inconsistencies may be incorporated into gameplay (where the player must notice and respond to an anomaly in order to complete game objectives). Also, games are by definition "less serious" (even for serious games) than many of the important productivity applications one would expect these heuristics to be the focus of. That is to say, games are almost an application class by themselves - productive work takes a backseat to having fun, which shakes things up a bit. It would be interesting to know if there has been any research done on heuristics specifically for games. I would expect that games are a heuristics class unto themselves, and that different types of games (RPG, MMPORG, first-person shooters, educational games for specific disciplines, etc.) demand vastly different heuristics than others.
Juanpadilla 16:25, 24 September 2008 (UTC)
The thing that really got my attention was the potential loss in revenues for each usability issue... With this in mind, one would think that it is absolutely necessary to employ this kind of analysis for a game company given the potential cost of creating the game itself. Moreover, the practice of following the list for the game industry is probably pretty difficult. For example, the one to one mapping of real world things with the actions of the game. I think this has been a HUGE focus in the game industry and while there has been great gains it has been mostly in standardized controllers. The new generation games like the Wii may be taking this issue in the right direction with motion detection and a focus on getting the user moving like the characters in the game.
Shyam Vijayakumar 16:54, 24 September 2008 (UTC)
At first when I read the discussion question, I thought that there might be some usability problems in the list that will not be applicable to serious games. But, it looks like all of the problems can be found in serious games. There are, however, specific problems with serious games like perhaps the orientation of the controls of the game that list doesn’t contain. This simply reinforces Nielson’s statement that heuristics must be applied in conjunction with user testing. Many of the minor problems that heuristics missed can be found through user testing. In the design-prototype-evaluate system we have, Nielson’s heuristics approach can easily be placed in the evaluation part along with all the user testing.
James Yeh 16:57, 24 September 2008 (UTC)
While Nielsen’s heuristic evaluation formula serves as a general guideline for assessing all products, the unique interfaces of game designs might require more specific problem checking to smooth out genre-related issues. The same principles mentioned in the reading still apply; however, some heuristics such as “aesthetic and minimalist design” and “match between system and the real world” might not be pertinent to every game. For example, a fantasy-themed rpg takes place in a vast and vivid world and usually involved numerous imaginary characters and monsters; in this case, the complexity of the user interface may be interpreted by an uninformed heuristic evaluator as “unnecessary” and “cluttered”. Nevertheless, Nielsen’s descriptions of common problems acts as a basic reference point that can be tailored to accommodate for a particular type of design.
Anthony Kilman 17:02, 24 September 2008 (UTC)
The Neilsen reading brings to light another form of testing useful in the development iteration process. Heuristic evaltuation provides a means to evaulate usability of a particular project. Although I feel as if his points are valid in regards to many main stream applications where the course of action is clear and defined, I have to agree with other students in that these don't cleanly apply to many popular games of today. There are clear cut interactions between say WoW (not a gamer, so forgive me if I make any incorrect references), and common key patterns or mouse patterns; but they are not the same heuristics that would be obtained with an application such as Notepad. One could garner statistics, but extrapolating usability patterns seems fairly difficult. Aside from that, a few things stood out about the evaluation process in particular: "debreifing" and the inability of the evaluators to talk during the session. I like the idea of the user having absolute control so the results are entirely based off of the user.
MuQing Jing 17:10, 24 September 2008 (UTC)
I feel that many of the heuristic aspects that Neilsen brings up are relevant to games. For example, the features of redoing and undoing serve important roles in games; in particular, serious games which are more focused on accomplishing a certain task rather than having the user solve everything perfectly. Also, the game's learning curve should be flat enough that both new inexperienced players and experienced players can get satisfaction from the game. There are some things that don't fit in with games though. For example, inconsistencies in games might actually be necessary; games that require puzzle-solving or observations will need some sort of fairly obviously inconsistency to draw the player's attention.
Of course, all this holds true only if it is applied a fairly low-level. At a higher level (once you eliminated gameplay), almost all of these aspects are true. The conceptual design of games and other applications are very similar in that since they all stem from code, they all share the same base and limitations, which the heuristics seem to address pretty well.
nathanyan 17:35, 24 September 2008 (UTC)
I can see how heuristic evaluation mentioned in the first reading could be useful in the earlier stages of development, where there are bound to be many issues and the goal might be to simply record them, and move on to find more issues. I'm not sure if it's more effective than ordinary user testing for a more developed/polished product however, particularly with respect to helping the evaluator. Interfering with the evaluation process and helping the evaluator effectively is a crutch for the design of the program - instead of forcing the user to run into and overcome problems on their own, helping the user simply introduces a shortcut and thereby bypasses any deeper issues that may arise with that usability problem. It's similar to "eating your own dogfood" in software development - if you're developing a web browser, for instance, sure you can use an alternative browser while yours is in development, since it will probably be more usable or won't have as many bugs. But if you've already got a stable version of the application you're developing, you should really stick to using it exclusively - even if there are more issues and you can't get work done as fast as a developed/polished alternative, doing so forces you to not only experience all the problems in the application, but delve deeper into how they may be solved, rather than simply recording them down as a to-do list to correct.
Mikeboulos 03:03, 25 September 2008 (UTC)
Nielsen's usability heuristics might not all work when designing serious games, for example: documentation, for a person playing a serious game, that might seem pointless. If I play a game and I need to go read the manual, then there is a higher chance of going back to read the manual again and again. which will make it impossible to play the game, the game need to evolve around the user and not the user around the game. Also recovering from errors, if a game has a glitch then playing the game, or saving the game up to that point might result with the same error, which might not be a good idea, also if we save the game at a lower level, will frustrate the user, however, some heuristics should be used so that we can have the first prototype done, or we'll be lost and try to create a generic game for all kinds of users, so heuristics are good to build on if they apply.
