Quantitative Evaluation

From CS 160 Fall 2010

Jump to: navigation, search

pdf Slides

Readings

Required:


Optional:

Your Response

Login by using your user name and password, edit the page and at the end of it put a whole line == ~~~~ == (literally) at the beginning of your submitted critique, so the wiki system will index, sign and date your submission automatically.

Contents


Frank Chew 02:03, 16 October 2010 (CEST)

Having a checklist of common problem types is a critical cycle of testing for the most important problem types in an interface. The author created a valuable list of factors to test for, and in which issues can be categorized. In addition to the items listed by the author, there should also be a growing list of other problem categories that are common and to look for when testing systems. It is interesting that the act of testing can change results of the tests in the future. There are additional experiments to run on order to measure the effect that the first test had, such as testing to a control group that did not take the test before. It's also important that variables have to be randomized that might otherwise be correlated. For example, making sure that there were as many fast-paced lectures on Mondays than on Fridays in order to ensure sure that the day of the week was not affecting the results of each pace rate.

Chris Song 03:43, 17 October 2010 (CEST)

Heuristic evaluation seems like an excellent way to minimize the amount of actual user testings. I was pleasantly surprised to read about Ten Usability Heuristics because I initially held the idea that there is no real way to set a usability rule since all systems are different. After reading through the article, I realized how a similar problem can plague many different systems. And then, it became very clear to me that through heuristic evaluations a designer must get rid of all the “obvious” problems. Even just looking at 10 usability heuristics, I was able to spot some of the flaws in our initial group project designs. Of course, we already made many fixes after contextual inquiries, but had I know about these evaluation criteria before, I would have been able to fix some of the obvious mistakes even before the interview. In conclusion, what I am trying to say is that we as designers should rely on every possible testing tools available to us, such as CI, heuristics evaluation, and user testing. However, it would be more beneficial if we have a clear idea of these guidelines even before the design. In other words, you could simulate performing a heuristics evaluation on your ideas before you ever start designing! That way you don't just put out a prototype and say, “well, someone will find flaws and I will fix it then.” Instead you can have a system that is already relatively error free and then through testing make it even better, not spend time fixing obvious flaws that shouldn't have been there in the first place.

Robert Connick 06:33, 17 October 2010 (CEST)

I wonder if working on minor problems in every iteration might be a waste of time. Since the ealrier changes will probably be coarser, the small problems could get swallowed up by fixing the larger ones, or new small problems could be created. Evaluators are not ordinary users, so they could concentrate on large interface issues first, and smaller ones later without being distracted too much by those minor issues. It would basically be the same idea as building a paper prototype first, speaking of which I do not see how a paper prototype (such as the one we built) could work here, since that would greatly increase the overhead of an evaluation session. The "paper prototype" that the reading mentioned did not sound like the kind we built though, since the evaluator could "turn the page," which makes it sound like it's just a sketchbook. The explanation of random variables was poor and I suspect false. I'm not sure how much of the rest of that reading to trust, but most of it seemed pretty reasonable and essential to know when experimenting.

Sean Tai 07:58, 17 October 2010 (CEST)

In the first article, I liked the heuristic of “error prevention.” As the reading points out, either making it unlikely for a user to make an error or having a confirmation dialog for potentially problematic actions make for a much smoother, pleasant user experience.

I didn’t find anything in the second reading that seemed specific to psychology experiments, but rather just definitions of concepts found in all scientific experiments, and thus didn’t learn anything new from it.

Brian Maissy 10:54, 17 October 2010 (CEST)

The method of heuristic usability evaluation makes a lot of sense, and seems simple enough. I am impressed by the author's research into the effectiveness and cost value of this method, and data which yields useful information, such as numbers of people to use for heuristic evaluation and rating severity of usability issues. The article on technology transfer of this method makes a good case for its effectiveness and points out some interesting issues about its application.

The second reading was mostly general concepts of experiments that I knew already. Some of the threats to internal validity I had not considered before. Part of responsible experimentation is admitting the confounding variables that cannot be controlled. Their qualitative effects may be able to be taken into account when analyzing the results.


Chao liu 14:13, 17 October 2010 (CEST)

As article mentioned, the heuristic evaluation gives us a good method to evaluate our UI design. A good heuristic evaluation needs certain amount of evaluators. And each evaluator can provide special opinions of the UI design and give their own opinions. I notice that there are some people that good at find the usability problems while some don’t. However, as the graph showed in the article, some special problems may find by some “unsuccessful” evaluators. Those articles give us a basic principle of how to conduct a qualified evaluation. Some basic principles are quite useful. I think in the next interview we are should follow those principles in the article “ten usability heuristics”.

Calvin Wang 23:23, 17 October 2010 (CEST)

What an eye-opening reading! I never knew that there were such a variety of usability methods, each serving its purpose. Heuristic evaluation is not as accurate and insightful as actual user testing, but its cost effectiveness clearly blows user testing out of the water. I understood most of the reading, but why is the ratio, not the difference, of the benefit and the cost used in determining the optimal number of evaluators in heuristic evaluation? Isn't profit the difference between the two?

The second reading is a nice summary of how to design experiments with good external and internal validity, but basically everything we're familiar with from AP Statistics. One important feature that's missing is the concept of a lurking variable. For example, studies have shown a positive correlation between ice cream consumption and occurrence of violent crimes. This sounds incredible until one realizes that the two are connected to a common cause - temperature. Although this isn't exactly a controlled experiment, lurking variables are relevant to inferring causation nonetheless.

Jonathan Look 00:52, 18 October 2010 (CEST)

For the Nielson readings, I thought the conclusions of his questionnaire were interesting, but I would be curious to know what the results would be if such a questionnaire were done now. I personally think it would be quite different, seeing advancements in the UI realm. The heuristics for evaluation of designs I think are useful guidelines for development and can help steer a project in the right direction.

I felt like the Martin article was very much a review of science class back in the day. I did like the discussion on confounding variables however. The example with the Coke and Pepsi test just goes to show how tests can be biased. However, I believe one confounding variable that the author did not mention about his lecture pace experiment was the types of lecture material being covered. I feel that a topic that is boring, interesting, confusing, or abstract could affect attentiveness just like the letter M or Q on a cup.

Alex Aberle 05:16, 18 October 2010 (CEST)

I think heuristic evaluation is a great idea, especially when coupled with user testing. One method examines the user as they use the system. It expects the performance of tasks to speak for itself. This is useful for some types of problems, but not all. Heuristic evaluation is direct elicitation: ask the evaluator to run through the interface and flag things that clash with a given list of heuristics. I think this is a great way of discovering problems a normal user study wouldn't. The fact that it is compatible with low-fidelity prototyping is a major plus, too. It allows you to get significant feedback without having to spend a lot of time on a high fidelity prototype.

Also, a big thumbs up to the author of these articles. They were clear, concise, and informative. Completely the opposite of last time's readings.

I found the second reading more interesting than I thought it would be. It turns out it's pretty hard to run an experiment and say for sure that your results are 100% valid. The issues with selection bias, differential mortality, and statistical regression seem the most insidious. A researcher would need to have a keen eye on those issues and a rock-solid way of accounting for them in the final analysis. I never really knew there were so many issues that could confound an experiment, but I guess thats why they pay scientists the big bucks (hahah, not really. Inexplicably, programmers get paid out the wazoo for work far less rigorous and useful than the work any scientist does)

Karl He 05:44, 18 October 2010 (CEST)

Heuristic evaluation is a very simple concept. It's like asking people to proofread your essay, the more perspectives you have, the more problems you are going to catch. The crux of heuristic evaluation is that you should pick evaluators to whom the interface is not familiar. Give these random people a list of what to look out for. This way, there is no inherent bias about how things should be, or what users are already accustomed to, but strictly usability problems. The entire idea strikes me as a "well, duh" kind of thing, but it is nevertheless important.

Andy Lin 06:47, 18 October 2010 (CEST)

The first article talks about heuristic evaluation, which is another good way to test the usability of our user interface design. I personally believe that one good thing for heuristic evaluation compared to user testing is its speed of finding usability problems, especially for the interface that is not hard to understand. The ten heuristics are really good ones to check, although it might be hard for programmers to translate each error message into plain language.

The second article gives clear definitions of variables used for design experiment. However, it is not really clear how it will be used in this class from the examples provided. I suppose it can be used for forming a model and do the analysis.

Raymond Williams 08:24, 18 October 2010 (CEST)

The "Consistency and Standards" heuristic [from the Ten Usability Heuristics page] says to stay with platform conventions. While I agree that this is a good idea, we must remember that it's not always the best idea to do something just because everyone else is doing it. It is entirely possible to come up with something new that may even be better than the current standards and conventions. Take, for example, Nintendo. There was no competing with Sony's PS3 and Microsoft's XBOX360, so they came up with something entirely new [the Wii]. If they tried to keep the conventional handheld controller, they might have gone out of business by now. Convention should be a guide, not a rule. Just a good way to know what's currently acceptable. I wouldn't try to fit 10 icons on the screen when everyone else is using 4, but I might change the way the user inputs text...

The reading by Martin had a nice section on variables [i.e. Control Variables, Random Variables, Confounding Variables, etc.]. I love to test my work. My goal is always to try to break my program through normal use. Then I try to break it with extreme use, exceptions, stress testing, and more. The problem is that it's hard to predict the configuration that the users will have. Often, my software will work fine on the systems on which I've tested, but some user reports an error that's due to a specific system configuration that I couldn't predict. I always find it frustrating to get my software to work on everything. I think that as long as there are many different platforms and operating systems with no shared standards or conventions, then this will always be a problem. Sometimes I have to make separate versions of my software just so it will work on a variety of platforms. This is unfortunate...

Bernard_Wong 09:56, 18 October 2010 (CEST)

Designing is an exploring process, people discover what works and what doesn't. Not all design works, designers often need to experiment with various designs, and test them with either heuristic eval, or user testing. Whether to say that design to stay to convention or heuristic, it's better to say designs should stay simple.

Anthony Puccinelli 10:19, 18 October 2010 (CEST)

The Heuristic Evaluation reading brought up some good points, not least among them how important it is not to have evaluators discuss and smooth over their response to your system with each other before getting each of their individual reactive responses. Otherwise, something like this results. Another good point was that lo-fi prototypes really are a perfect choice for this type of testing because the testing is focused on usability, not functionality. Both articles gave a lot of good advice concerning how to test/potential pitfalls of valid testing. The idea of treating user testing as a scientific experiment is interesting, and we would do well to adopt it for our upcoming assignment.

Melissa Lim 10:37, 18 October 2010 (CEST)

The first link on Heuristic Evaluation advised for 3-5 experts to evaluate an interface. However, there was no specification on who these experts should be. I think it is important for these people to be professionals who either specialize in a specific area of interface evaluation or who come from different backgrounds. This helps to have a broader spectrum of ideas and will most likely lead to uncovering more problems. Overall, I believe Heuristic Evaluation is a helpful tool after the prototyping stage if the experts hired have a good understanding of the target user group and also account for the edge cases- for example, making sure the site is accessible by handicapped users who rely on screen readers. It would have also been interesting to read about how one becomes an "expert" as well as a walk-through of this expert's thought-process when evaluating an interface.


Daniel Yoo 10:51, 18 October 2010 (CEST)

I think Heuristic evaluation is a great way to inspect the user interface design for usability. The author gives ten usability heuristics with specific guidelines, and I am quite amazed by that because people have put lots of inputs on this topic and out of many examples there are ten that have been selected. Heuristic evaluation helps me realize what I should do for our group assignment design and make a better evaluation with a user interface. Another example is also related to design experiment with independent and dependent variable and internal and external validity. This reading is more like a research paper that the author gives example of selection, morality, and statistical regression.

Tiago Bandeira 13:28, 18 October 2010 (CEST)

Heuristic evaluation is a very useful means of testing an interface prior to testing it with actual users as a way to reduce costs by finding any blatant errors early. Many professionals like using evaluators but the emphasis is still on probable users since most companies in the survey used only an average of 2 evaluators while many used as many as 10 potential users to test the UI. A question I have with this reading is about how the author cited the ideal number of evaluators to be at the point where the benefits per evaluator are the highest. However, I think that it would be optimal for a company to continue hiring evaluators until it would be cheaper to release the UI with any potential problems it might have given the evaluation.

The second reading on testing was interesting especially since the subtext is that psychologists want to be seen more as scientists rather than be associated with the humanities fields. Regardless, the text did a good job of explaining why randomizing as many of the independent variables as possible is so important. However, since most UI tests are done with very few people and the tests done are not physiology tests I’m not sure how applicable the information is.

Alan.choi 14:00, 18 October 2010 (CEST)

I find that list of ten reccomended usage heuristics to be a dose of common sense, but also a very valuable resource as it is hard to really sit down and think about what exactly it is that we deem good usage. The list is fairly comprehensive and really allows people to effectively point out issues or even design it in the first place.

For the second reading, it was mostly a summary of basic statistics and experimentation procedure. However, one thing that intrigued me was the concept of experiment mortality. It had never really come to mind before, but the act of quitting an experiment can be part of the experiment as well as they gave an example of the stressed managers.

Bichen Wang 14:27, 18 October 2010 (CEST)

Using the heuristic evaluation approach to improve a user interface seems like a cost-effective and efficient way to handle UI design. Although the study still shows user testing is the best way to test a product, heuristic evaluation is significantly less costly in terms of time and money and yields almost as good results. Since only anywhere between three and five evaluators are needed to find most of the usability problems in an application, it would be interesting if one group could evaluate another group’s app. This way with four or five people evaluating one app, a new set of eyes with this new heuristic evaluation would definitely benefit all of our apps. Since direct user testing still is a little better than heuristic evaluation, the second article explains just about everything that can happen. What really is interesting is how it notes that experiments should only control a few of the more important things. Having results in a computer lab with cameras set up is probably very different when the user actually uses it. However, there are some things that cannot be replicated, and randomizing with just three testers is a bit difficult. This definitely pushes towards higher-fidelity prototyping and a more rigorous testing in the correct environment.

Samantha Paras 17:29, 18 October 2010 (CEST)

Heuristic evaluation seems like an important counterpart to user testing. It sounds like a good way to iron out simple problems cheaply. While in user testing one just watches someone try to use their app and then gains insight from there, in heuristic evaluation there is more of a focus to find out about UI problems. The ten example heuristics given seem a bit obvious but I know I would not think of all of these things if told to write them down (such as "Recognition rather than recall" -- making sure the user doesn't have to depend on their memory, and instead displaying everything).

I believe a lot of us already know about running experiments though science classes in high school, but it was nice to have a review of the basics such as independent vs. dependent vs. condition variables.

Edmundo 18:36, 18 October 2010 (CEST)

Heuristic evaluation seems extremely useful as it is presented in the reading. I especially like the simplicity of it as well as its focus on finding as many different issues as possible. The fact that no actual users have to be used seems to make the process that much more viable because one does not have to go through the process of looking for users which in some projects might be extremely difficult or expensive. The greatest effect that doing a heuristic evaluation has on a subsequent user test is probably that the users are more able to give feedback that is more focused on their tasks and also feedback that pertains to their expertise. Lastly, having such a well-defined checklist and having every problem separately documented must also help in classifying and specifying the different problems.

The second article was interesting but I feel like I am missing why we are reading it.

Christine Lu 18:58, 18 October 2010 (CEST)

I particularly liked today's reading because it gave step-by-step instructions on how to perform a heuristic evaluation and pointed out both the pros and cons, as well as gave details on the minor aspects, such as severity ratings, rather than just mentioning them but not explaining what they are. I thought it was interesting that the article pointed out that in lo-fidelity prototypes, users aren't as to finding missing elements, because the aspect of being a paper prototype means that the user can bypass difficult steps by physically moving elements around, which isn't as helpful to the evaluator, especially if this confusion isn't observed or noted. However, for actual running prototypes, the user has no choice but to stall and try and figure out a solution because the software makes it such that the user cannot go on. The second article was a good review about how to conduct experiments and the difficulties in doing so.


Richard Laroue 19:00, 18 October 2010 (CEST)

Today's reading on heuristic evaluation introduced a very useful new tool for evaluating interface design. Taking these simple "rules of thumb" into consideration when designing an interface can go a long way. For instance, just knowing that users do not often like to feel like they have no options or feeling like they are trapped can suggest that designers should provide "Cancel" or "Back" buttons. Also, designers can more effectively make error messages in a way consistent with how the user expects or would like to see these messages by following some of these heuristics. Our group will likely consider these heuristic when making our design as well.

Mark Wei 19:37, 18 October 2010 (CEST)

The 10 Usability Heuristics is definitely a great thing to keep in mind. I feel like it would help if every application designer had this list nearby when they do their design, since it just seems so useful. I have been frustrated by many of the mentioned points, like the lack of a cancel or back button for some task that makes you go out of your way when you accidentally press something. Another example if the suggestion to use readable error codes. So many applications fail to give you an insightful error message, and instead just dumps a huge stack trace or Exception message that you don't know what to do with.

Don Arboleda 20:07, 18 October 2010 (CEST)

The idea of confounding variables interests me. "The best laid schemes of Mice and Men oft go awry," as the saying goes; there are so many things that can go wrong with experiments that, often, we miss the small things that could completely destroy the credibility of our tests. The example given about Coca-Cola and Pepsi-Cola in particular also fascinates me. I've heard people claim that they can tell the difference between the tastes of Coke and Pepsi, but my tongue (apparently) does not have that sort of resolution for colas. I've contemplated running some sort of double blind test on my friends just to see if they can put their money where their mouths are, but I haven't found the time. As the example implies, however, most people actually can't tell the difference between the flavors at all.

Richard Nguyen 20:08, 18 October 2010 (CEST)

I really liked the idea of heuristic evaluation. It allows designers to fix obvious errors early on in the process so that when it finally does get to testing with real users, we can use those user tests to weed out more important problems. Not only does it seem to make the testing phase a little faster, but it also would reduce costs since we might require less trials with users. The fact that it can be done with 3-5 experts makes it very doable quickly and cheaply. The 10 usability heuristics list is also a nice resource to have on hand to keep the designers focused on what's needed, because even though they seem kind of obvious, things like "recognition rather than recall" might slip a designers mind especially if they already know so much about the system/program that they're designing.

Alexander Wong 20:19, 18 October 2010 (CEST)

Compared to qualitative evaluation, quantitative evaluation seems to give you more bang for your buck. Obviously this was brought up in the reading in some detail (fixed and variable cost with utility maximization), but it is also true in terms of time spent. Quantitative evaluations seems like a perfect complement to low-fidelity testing and the push towards multiple iterations.

I particularly like the idea of presenting the evaluator with a set of heuristics. This does prime their experience and the type of usability problems they may find. If they are presented with an incomplete list they may be influenced and overlook usability problems, but if you present them with a complete list the priming could help they find problems they may not have found before. Particularly for amateur evaluators I could see the heuristics being a great help. Furthermore, the heuristics help code the usability problems you find into concise and relatively discrete findings. Going back to improve the interface should be as simple as looking at the coded list of problems, with each category suggesting a particular type of fix; that is, if a problem is coded as a consistence problem, your domain of fixes is limited to those relating to consistency. The search space is smaller and it is easier to create incremental improvements.

Sui Kun Guan 20:26, 18 October 2010 (CEST)

In the articles linked by "Heuristic Evaluation," not only having the evaluation of the actual users is important, the Heuristic Evaluation is also important when discovering the user interface design problems. When doing Heuristic Evaluation, it is recommended that before discussing the interface problems together, writing down own opinions for each evaluator individually first. Also, having more evaluators can discover more interface problems more easily. Therefore, Heuristic Evaluation should apply to our group's project.

In the article "How To Do Experiments," when designers conducting their experiments, in order to get a more general result, having less controlling variables will be better. On the other hand, controlling variables are necessary because designers usually want to know how any changes on variables will affect the changes on independent variables while keeping all other dependent variables to be constant ideally.

Aaron Loessberg-Zahl 20:29, 18 October 2010 (CEST)

The heuristics article will be infinitely useful in performing testing of our interfaces. The list of heuristics is probably the most important part, but another important, perhaps overlooked, point is that different usability problems come to light when using a paper prototype versus a realistic prototype. Although this does not apply to us, I was amazed at the cost-effectiveness of heuristic evaluation.

The second reading was nothing new to me, primarily because we read almost the exact same thing, albeit in a textbook, in the first week of Introductory Psychology. There were, however, a few new concepts I hadn't covered, including differential mortality and statistical regression. Overall, I feel that this article, although very good, tries to be too friendly with the reader, and suffers a loss of precision in its message as a result.

Courtney Wang 21:00, 18 October 2010 (CEST)

The use of heuristics in evaluation during model testing is much more intuitive to the developer with a technical mindset. Most computer scientists prefer to have numbers to work with, and heuristics/quantitative evaluation provides a numerical framework for analysis. It gives a quick and simple way to quickly iterate through early prototypes so that it can be refined in shorter cycles before going to qualitative evaluation. I think a multi-tiered development process where quantitative evaluation is done in the early stages to get obvious problems exposed and solved before qualitative evaluation is done on refined prototypes leads to well-rounded and efficient development. The article was extremely useful by providing a list of heuristics for us to use, and also a step-by-step guide on how to evaluate qualitatively. I am relieved that there is an established method to quantitatively approach user interface design; a perfect design can't be done with only heuristic evaluation, but it provides a way to quickly refine the early iterations before spending a lot of time with users.

Soroosh Izadian 21:02, 18 October 2010 (CEST)

I think Heuristic Evaluation is by far the most efficient and systematic way to evaluate user interface that we've read about. The ten guidelines presented on the article allow the evaluators and observer to rapidly identify the "obvious" flaws in the design. This paves the way for user tests and field testing to be more precise and detailed by taking care of the most common problems earlier. The rating system for Heuristic Evaluation also helps to prioritize the fixes based on the severity of the corresponding problems. Meeting deadlines is so much easier this way since the developers can apply fixes in a very organized fashion.

Asa Zernik 21:04, 18 October 2010 (CEST)

Heuristic testing seems essentially to be a group design review; the essential new points in the reading seem to be the emphasis on individual reviews (each person finds different types of problems, and you don't want to lose the unique observations by putting people in groups) and the specific observations about what kind of interfaces are in reality hard to use.

Karthik Jagadeesh 21:22, 18 October 2010 (CEST)

The Ten Usability Heuristics were an interesting compilation of the general principles for the user interface design. At first many of these guidelines seem obvious and trivial, but it is actually useful to think about these different points when building the application. Since application developers are usually think about building the main part of the of the app, they might forget about these minor details which make a big difference to the user.

The Heuristic Evaluation seems like a good alternative to the methods for usability testing that we learned about earlier. It seems relatively cheaper to do, and takes much less time, just a couple hours. Also it can be very hard to find users, so this method will allow a few number of testers to find most of the major flaws, and the gains from having more testers is minimal. Also, the other methods that we learned about were indirect where we observed the user and tried to infer things based on their behavior. This method is much more direct, and takes advantage of research made by thousands of other people to know exactly where to look when evaluating a prototype.

The second article brought up the interesting point of maturation. When you are conducting tests, and if the product is meant for beginners, the more that the tester plays around with the application, the less of a beginner she/he becomes. I think the way to solve this is to continuously change out the testers after each iteration. The article looks at a conducting experiments from a highly scientific perspective. This is really helpful, and will cause the reader to think about different things to take into account when making an experiment.

James Yu 21:24, 18 October 2010 (CEST)

Heuristic Evaluation seems to be a good way to detect "obvious" errors in a design. It seems to be a more systematic approach than normal user testing or contextual inquiry and may catch some things user testing may miss. It also seems to work well even when generalized to many different systems, and shows there are usability problems that show up in many different systems. The readings did say missing component problems may not be discovered in paper prototypes, so that seems to be something to look out for. Overall it seemed like a useful testing method to use for our project.

For the experiment reading, it gave me some more insight on the experimental process. I was aware of independent and dependent variables, as well as controls, but I haven't heard of random variables and I've never really understood external variability even though I have heard of it. Some times I see experiments where there are obvious controls that haven't been applied, but given the introduction to random variables, now see why sometimes it is okay not to apply controls to some variables.

Sung Ma 21:27, 18 October 2010 (CEST)

I liked the article on Heuristic evaluation and it seems very useful when finding problems and errors. With the ten usability heuristic guidelines, it can greatly reduce cost and time with use of few experts. Although the word 'expert' seem too vague, this method can lead us to better application when we are working on our projects and think more about the user interactions. I believe that Heuristic evaluation is one of many methods out there that we can work with. More tools we have better projects we are going to get. Knowing various method of development is important.

Benjamin Carpenter 21:30, 18 October 2010 (CEST)

Heuristic evaluation seems like it works really well for products that will be used by a large number of customers/users. It may be harder and more expensive to find evaulators for very specialized systems, that are designed for use by people with highly domain-specific knowledge. Even still, because you only are using around 3-5 people for the evaluation, it could easily be feasibly to apply this type of evaluation to specific domains, as you can most likely find that many experts. It would also be interesting to observer how much the evaluators notice the heuristical problems, as certain evaluators may be pickier and more receptive to problems, while others may have a higher tolerance for frustration.


Danica Shei 21:32, 18 October 2010 (CEST)

Heuristics seems to be a very cost efficient way for designers to improve and reconstruct their product quickly and efficiently. The list of ten heuristics again re-emphasized how the success of certain products depend largely on the usability of the user interface and less significantly on the actual product. Although I first believed that the more evaluators you have the more ideas and improvements would be generated, it makes sense that the optimal number of evaluators would be four. This amount of evaluators encourage group interaction and exchange of ideas - and would be a vast improvement to their original individual analysis.

Yue Chang Hu 21:38, 18 October 2010 (CEST)

The first article, "Heuristic Evaluation" gives a pretty good guideline for interface designer to follow, especially the 10 heuristic. It argues and i do agree with it for that heuristic evaluation of the interface design is good because it can spot strength and weaknesses of a design. The second article, is just a list of statistical tools for studying and analyzing a good interface design that most of us probably already learn something about it from cs70. Nevertheless, something that interest me is internal validity which refers to whether a change in the independent variable or something else caused a changed in the overall study. The author provides a list of internal validity such as historical, maturation, selection,mortality, testing, statistical regression to remind..etc to remind us to be more careful when doing an experiment.

Jeremy Sasson 21:40, 18 October 2010 (CEST)

I like the idea of switching between Heuristic Evaluation and User Testing. Although the two may seem similar, they in fact do very different things. The Heuristic evaluation is just there to make sure that everything fits together smoothly, that the buttons are in the right places and that the interface looks good. User testing is more geared towards making sure the functionality of the product is set. Alternating between the two of these during the iterative process is the best bet for a usable, functional product. I believe Heuristic Evaluation is much more useful than ethnomethodology in that it allows you concrete data to work with, while ethnomethodology is much more abstract.

Derrick Tao 21:40, 18 October 2010 (CEST)

Heuristic evaluation seems to be a very useful idea that can be used to find potential problems and errors. I am not really sure what it means when it says to find 5-6 experts. The term experts is a bit vague. It is important to make sure that these experts are experts in the field of user interface and design yet also come from different backgrounds to make sure that they cover a wider view while evaluating the product. With good experts, this could really help the developers because it is easy for them to sometimes forget minor details of the project.

Arthur Huang 21:46, 18 October 2010 (CEST)

I liked the article "Heuristic Evaluation" because it provided a simple and clear guideline on how to easily and cost-effectively improve and test UI design. The best part is that it does not involve an actual user, so the designer does not have to worry about finding a subject to conduct the tests on. The ten examples are kind of obvious and seem pretty much like common sense, though. Still, I find this to be potentially very useful for our final project design.

Robin Liu 21:48, 18 October 2010 (CEST)

Heuristic design seems to be a great way to abstract the elements of interface design into a general set of criteria that is applicable independent of the product at hand. It allows the designer to encapsulate the various dimensions of usability in a quantitative manner, and provides accurate and scalable feedback.

Seng Heng 21:58, 18 October 2010 (CEST)

Quantitative evaluation seems a lot cheaper and easier than qualitative evaluation, though you have to have evaluators that are trained to heuristically judge interfaces and find interface flaws (as opposed to just user testing with laymen). Finding trained evaluators may be more difficult than the reading makes it out to be, but it would produce more solid adjustments and interface design choices instead of the oddly abstract methods of ethnomethodology. This shouldn't replace user testing, however - it's easy to fall into a trap where you build to the specifications of the same group evaluations, as opposed to what the end-user on the market wants.


Vincent Rodriguez 22:17, 18 October 2010 (CEST)

The heuristic evaluation seems like a good way to test for properties of your product that you know are present in the product and want to test for. It's easier than contextual inquiry since each heuristic test is geared for testing a specific feature in your product, as opposed to the contextual inquiry method where you just sort of let the users roam wild with your product. The only thing that to me might be a bit of a problem is the fact that you have to run a lot of tests in order to get something meaningful out of it, and this might take up time that might have otherwise be used for something else. As for the scientific method reading, I found it interesting to read about compounding variables, or those variables that could be interpreted differently by different users. That is, that a property attributed to a specific variable could also be attributed to something that could be a by product of just running the experiment. Although I feel as if the reading makes the scientific method less definite about its results because of all the added complexities of real life. Then again, I guess this makes it very realistic.

Avery Gee 22:47, 18 October 2010 (CEST)

Heuristic evaluation seems to be a good concept. While it might take some extra time, I think it's very important because the look and feel of applications is important to most users. The 10 heuristics are nice because they provide some basic components that every application should use. If everyone followed this it would really improve user experience. This is because then there would be some general components that are always there. This would help the user's mental concept of the application and expectations for those components would exist before going into the application. This makes navigating significantly easier, as the user already has a pretty good idea of what to do even before they open a new application.

Theron Ji 23:51, 18 October 2010 (CEST)

I found the first article extremely useful and illuminating. These heuristics all have very intuitive meanings that make them easy to understand and believe to be true, but yet are extremely applicable to almost all facets of user interface design. I have seen this used in real-world industry, and is taken very seriously, so this article has a very practical importance. The list of 10 is a useful cheat sheet to have handy, especially when the UI problems are more subtle and its good to have a checklist of things to check for. It's surprising how simple, yet effective and useful these are. The second article strikes me as interesting because the methods it uses are very similar to the scientific method taught in basic science courses. The approach draws heavily on quantitative scientific methods, so it brings a much-needed quantitative side to this whole, very qualitative field.

Retrieved from "http://bid.berkeley.edu/cs160-fall10/index.php/Quantitative_Evaluation"
Personal tools