Qualitative Evaluation

From CS 160 Fall 2008

Jump to: navigation, search

Lecture on Oct 15, 2008

slides

Readings

and (optional)

Discussions

Please post your critiques/commments on the required readings below. To do that, first login by using your user name and password, then click the "edit" tab on the top part of this page (between the "discussion" page and the "history" page), New to wikis? Read the Wiki editing guide. . Hint - Please put a whole line == ~~~~ == (literally) at the beginning of your submitted critique, so the wiki system will index, sign and date your submission automatically.

Contents


Perry Lee 16:30, 14 October 2008 (UTC)

After reading the two articles, I was left feeling ethnography is little different from contextual analysis in purpose, but equally important: both seek to understand its users, just on a difference of scale and perspective. Whereas contextual analysis seems much more focused in its scope (at least in its approach of observing and talking to individuals as they work on their tasks), ethnography seems to approach the issue from a much larger scale -- it tries to examine a user's culture, not just the tasks he or she is trying to accomplish. "Everything" is considered important and written down in the hopes it will provide useful information. It makes sense to me -- the environment in which an individual works certainly affects him or her.

James Yeh 20:08, 14 October 2008 (UTC)

Ethnography appears to be a more generalized and flexible approach towards gathering information than the other methods we have encountered so far, but nevertheless serves as another tool for the designer to create a product that can better fit the users’ needs. The main difference with ethnography is that there is no specific task or item that is being focused on; whereas techniques such as prototype testing and heuristic evaluation provide a particular product to be tested or specific problems to be looked for, ethnographers seems to have nothing in mind, choosing to obverse and learn rather than diagnose and analyze. As a result, it appears as if ethnography is probably better suited for the earlier stages of design, in which the prototype has not been built yet, and major design decisions have not been made. Due to the nature of the approach, many unknown factors or details could be revealed from ethnography; thus, the designer’s view of the target user could change after the test has been conducted. Overall, ethnography seems to be broad method that could certainly be useful in improving the design of a product; different companies/designers may just have to tailor the method into a more specific scope to suit their purposes (as was mentioned in the second reading, full-blown ethnography can be very slow).

Antony Setiawan 21:03, 14 October 2008 (UTC)

Ethnography, as it seems, try to improve the system use in the culture by finding all problem in the way it is currently used. Having said that, wouldn't ethnography works only for particular culture? Nevertheless, I believe that ethnography is indeed necessary because one can't just assume/ build a system based on what it has been, or overly generalize the user. In addition to that, I hardly found a system that works well for different cultures. Ethnograpic methods in the early design, as Burke mentions, is used for the reason of the the high risk of designing a system for an unknown or misunderstood user warrants the time investment in ethnographic methods.

Jimmy Nguyen 00:43, 15 October 2008 (UTC)

The HCI/ethnographic methods described in the first article seem like a great way to provide quality feedback during the design process. It is neat how these types of tests can reveal a lot about tasks and relationships, even with the tests about the tablet laptops. It even showed an interesting facet, that people use these tablet computers event for being lazy and messaging. I am not sure what value comes in this particular case, but it shows that this method is capable of showing interesting capabilities and usages of the product. From the second article, I think that the most important result extracted from this method/experiment is the causal workflow. All of these scenarios are real events based on the results of the user experiments. I think it's important because it shows what event leads to another, what CAUSES what and it really illustrates the thinking process, broken down and put into a diagram on paper.

Witton Chou 00:49, 15 October 2008 (UTC)

It's important to see how the user will interact with something in their natural environment. It seems that there is an emphasis on observing how the user's culture will impact his interaction and behavior. Combined with contextual analysis, ethnography can help explain why or how people use a particular product or particular tendencies they observe when performing a task. As a result, it may be possible to take advantage of these behaviors in user interface usability.

Gary Wu 01:51, 15 October 2008 (UTC)

Though there are disadvantages to using concurrent ethnography, I feel that the advantages and information gained from supporting it far outweighs them. The point I liked about evaluative ethnography is the idea of having a narrow focus. I feel that people get overly ambitious when thinking about the bigger picture. Having a narrow focus helps the group, as well as the user, concentrate on the evaluations of the prototype. I feel like the quick and dirty method of ethnography is applicable when trying to scale results and tests to a larger scale. Rapid ethnography actually seems like a better solution and will probably succeed its predecessor. I think that reexamining previous studies can only help in avoiding repeating mistakes from the past. With the rate that technological culture is changing, I can see how quickly these studies become out of date. The new rapid prototyping systems mentioned in the articles seem very promising. The HCI methods (CUTA, CARDS, GOMS, PICTIVE) are all great examples of practical HCI development prototyping.

Jordan Berk 02:27, 15 October 2008 (UTC)

Given the various methods of performing ethnography described on the CHARM website, as well as the general descriptive content in the Millen article, it seems that finding the right balance between time and results is crucial to getting the amount of HCI feedback the designer needs. Rapid ethnography as described in detail in the latter and the "quick and dirty" briefly described in both, provide a way to get down big ideas and overall design elements. But on a smaller scale, they may be missing important elements that just can't be seen in the shorter timeframe these techniques allow. That's why concurrent ethnography and evaluative ethnography can be so valuable. To me, it seems a like combination of all of the above, used as necessary, would provide the optimal balance between results on all scales and time usage.

Anthony Kilman 02:58, 15 October 2008 (UTC)

Introduction to Ethnography: Ethnography is a branch of HCI research adapted from sociology and anthropology. It loosely refers to the act of ``observing human interactions in social settings and activities. This acts as a powerful assessment of users' needs. The types of data gathered in ethnographic studies can be broken down into two groups: (1) qualitative, and (2) quantitative. Additionally there are various methods by which ethnography can be integrated into user interface design: (1) concurrent ethnography: design and ethno data occur simultaneously, (2) evaluative ethnography: evaluation of a new design, (3) quick and dirty method: Usually a precursor to other methods. Increases awareness.

Rapid Ethnography: Paper presents a potential solution to the demand to spend time in the field while keeping pace with the 'ever-quickening' software development cycles. Three key elements to rapid ethnography: (1) Narrow the focus appropriately before the study. Prioritize important activities and choose the proper subjects accordingly. (2) Use mutliple observants to see exceptional user behavior. (3) Use collaborative and computerized iterative data analysis methods.

Kumar Garapaty 04:27, 15 October 2008 (UTC)

The two articles discussing ethnography depict it as a way of understanding the culture that you are building the product for and to improve HCI for this particular culture. In this way, the design of the product can then be focused on one particular culture to provide for its needs better resulting in a more targeted end product. Since this ethnographic data can be abstracted away from the actual design of the project, this information can be shared and used for designing a wide range of applications.

Hao Luo 04:30, 15 October 2008 (UTC)

The ethnography methods described in the readings sound almost exactly like the other interviewing/user-testing methods we have encountered. The designed is supposed to study a user in a natural environment using the product and analyze how the user interacts with the product. The readings describe different ethnography methods and ways of carrying them out as if it were some complex field of study with many different and valuable ways of approaching it. A closer look reveals these methods to be quite banal. A "quick-and-dirty" method which is performed with time constraints? You mean, you can perform a quick version of ethnography in order to save time but sacrifice deeper analysis? Who was the creative mind behind that gem? Other revelations include improving your feedback if you *gasp* use more than one researcher! In conclusion, I feel like I have learned very little from these readings which seem like a poorly-written rehash of other readings with superior user-testing methods.

Buda Chiou 04:48, 15 October 2008 (UTC)

Ethnography is really similiar to the context Inquiry we have seen before, but I think ethnography is more reliable for designing user interface. The reason is that we only interview a few people whom we think can represent the target user group for the program, which means we can only get the information from the interviewee so that the result won't be very general. However, since ethography more tend to inspect the background of the target user group, the result would be more general and capable of presenting the whole group of people.

Mohammed Ali 05:15, 15 October 2008 (UTC)

The three areas of the ethnography, contextual inquiry, observational study, and Participant observation are vital in understanding the viewpoint of the user. This is important because it is vital information that needs to be evaluated before designing or modifying a system. That is why I think it is important for ethnography studies be done very early in the design process so as to use the notions gathered in the studies effectively.

Xuexin Zhang 05:53, 15 October 2008 (UTC)

Personally, I believe that the ethnography method and context inquiry are essential parts of the Human Computer Interaction design process. During the context inquiry process, HCI designers could learn what's the user's tasks and how the user work on the task. In comparison to this, the ethnography method encourages HCI designers to research into user's culture and their customs during the design and improve usability of their products accordingly. By utilizing the ethnography method and context inquiry during their design, HCI designers could understand the targeted user group better and come up with more suitable design for the targeted user group.

Saliem Than 06:21, 15 October 2008 (UTC)

I had some thoughts about the second article that described ways to do "fast and dirty" ethnography and wondered if there might be anything described there that might lead one to generalize a particular segment of uses mentioned in the first article

Reading Notes:

"fast and dirty" (rapid ethnography) approach often practiced and favored due to time and money constraints.

Going to suggest the negatives with few solutions (bad practice, more solutions is better but eh.). Look for pitfalls. Not looking for the benefits of this approach. Maybe next time.

  1. See if they way they analyzed and conducted "fast and dirty" ethnography led to the pitfall of generalizations of the target market segment
  2. See if they had measures to prevent the generalizations
  3. Condensed Ethnographic Interview?
  4. Video observation - interesting - would have to be hidden and unknown to be effective (which would be difficult because the participants HAVE to know that they're being videotaped)

RAP - group think? too much interaction between the "team" leads to easy settling on market segment characteristics.

The Method: Focus and Key Informants

1. have to favor picking specific key "actionable" area of research (as opposed to wide angle, time consuming approach), this may lead them to assume too many things. So if this approach is taken, have to take into account the assumptions made. The strategy for picking informants is still not general nor random enough and subject to bias of the "team"

Interactive observations Collaborative Data Analysis

  1. I like the idea of visually rich analysis -

Case study

  1. thinking spaces? the way the internet would change the way we work? interesting.
  • focusing the observations and key informants
  • interactive observations
  • understanding field data


...this approach isn't too bad because it does practice in some form or another, the ethnographic techniques described in article one. I suppose when you need to, you have to choose based on whatever constraints you do or don't have, despite the pitfalls.

Haosi Chen 06:23, 15 October 2008 (UTC)

The HCI methods described in the first article seem like a really nice way to provide quality feedback during the design process. It is cool that how these types of tests can reveal a lot about tasks and relationships, even with the tests about the tablet laptops. It even showed an interesting facet that people use these tablet computers event for being lazy and messaging. The two articles discussing ethnography depict it as a way of understanding the culture that you are building the product for and to improve HCI for this particular culture. In this way, the design of the product can then be focused on one specific culture to provide for its needs better resulting in a more targeted end product.

JoshuaKwan 07:19, 15 October 2008 (UTC)

Ethnography is contextual analysis on a cultural scale. It describes the inclinations, on average, of a set of users unified by a single culture, a single overmind if you will that represents seemingly idiosyncratic behaviors that are a result of that culture's values. The role of ethnography is to determine what tasks are more important for a particular set of people to allow the developer to focus on those tasks.

Such idiosyncratic examples include:

  • many typical Americans are interested in sports, finance, and celebrity news
  • many Asian cultures are very family oriented
  • the Japanese love to use their cell phones for everything

etc.

A strong understanding of your target audience's cultural background can provide significant insights into what they really want, and in the case of lo-fi prototype experiments, can answer some puzzling questions.

Rapid ethnography in particular is just a way to reduce the expense of performing an ethnographic study by carefully choosing subjects and performing data analysis.

Wenda Zhao 07:30, 15 October 2008 (UTC)

David Millen's Rapid Ethnography is very helpful. It teached on many aspects of HCI including gathering user requirements, developing and etc. I really think that rapid ethnography can help us on understanding the users' thoughts and also some of the developing limitations. I also think that the case study also helped me understanding the rapid ethnography.

KevinFriedheim 08:12, 15 October 2008 (UTC)

I have to agree with Witton Chou in that you kinda have to see how someone is going to react in a given natural environment before you're able to predict behavior and behavior patterns. For example, someone growing up in New York Bronx might have very different trust issues than someone who grew up in say, Beverly Hills. But basic connections can, in fact be made (which is I think the point that Joshua Kwan was trying to make by his idiosyncratic examples. Its like he says, a strong understanding of your tart audience's background will ultimately be your selling point in the design process.

Vedran Pogacnik 08:33, 15 October 2008 (UTC)

I just don’t see rapid ethnography working in all cases. As the application has to be carefully designed in terms of software, the design of the application has to be in turn designed in terms of its users. Granted, since users are humans, there is an underlining connection, but when taking into perspective other factors, like availability of internet connections, average computer “quality” (that is, how good are the computers) in that particular region, and other similar factors, one would almost conclude that designing a “universal” app solely based on that underlining connection is a loss of resources. Thinking in a global frame, I don’t imagine someone in Africa of North Korea, who lives in a place without constant access to electricity, being compared with someone from the Bronx, and considering their needs and expectations with equal weight. That being said, I think the traditional models, the ones that target people specifically based on their profile, are better, despite the fact that they are very slow; the counterpart to that is that they are thorough.

Frank Yang 09:23, 15 October 2008 (UTC)

At first I thought the description of ethnography was extremely similar to the interviewing methods described before, such as the master/apprentice model. However, after reading closer, I saw that the goal of ethnography was much more in-depth. The whole idea of submersing yourself in another person's culture seems like a trivial thing to do at first, but I can see how understanding the culture behind your target user could make or break your interface. It would be extremely likely for a certain culture to expect certain things in everything they do in a certain way. For example, perhaps a culture might appreciate a toolbar on a different side of the screen, simply because they begin looking for things from the right side to the left, due to a different languages. Knowing how other people will react to your product is extremely hard without having input from users besides yourself.

Bing Wang 09:15, 15 October 2008 (UTC)

Ethno based user interface is definitely necessary as different cultures take different symbols and ideas differently. The first two article's study based on ethnic groups is especially important if the development team is from a different cultural background than the users. Some things that are intuitive in one culture might mean totally opposite in another culture. One of the examples that I can give is the meaning of colors to different cultures. The google finance pages for Chinese markets use red as up while green as down; however, the google finance pages for US and UK markets use red as down while green being up. This is clearly because Chinese culture of associating red with good. Without interviews of the local population with their culture, I think it would be difficult for one to figure out these issues.

The last article talked about a personal experience dealing with cultures. It shows not only the importance of cultural study but the big companies such as Intel is also doing them. It make sense that they are done in order to get a better model of the user population.

Alan McCreary 09:42, 15 October 2008 (UTC)

The general principle behind (rapid) ethnography for HCI is the same as the principle behind contextual inquiry - learn about the user as well as possible, resorting to speculation as little as possible, and going for direct observation instead. The idea of doing field work in teams seems particularly useful, since a team of researchers can observe more and make more connections between different findings. With master-apprentice contextual inquiry, on the other hand, the interviewer could miss some important aspects of the user.

Jacekmw 09:42, 15 October 2008 (UTC)

The concept of sociological ethnography pushed by the first and second papers is definitely important when it comes to user interfaces, as different cultures have different expectations and customs. For my group's game in particular, a game focused on food etiquette, ethnography plays a significant role. In fact, we initially were going to include many different types of restaurants, but eventually decided on a modern western high-class restaurant (French, in particular). However, it would be a very good idea for us to test the game with users of different cultural backgrounds, as differently cultural users may have different sticking points within the game that we may not expect. Thus, combining concurrent ethnography with contextual inquiry as a UI is produced leads to good UI choices in a culturally-focused game. The various rapid methods of ethnography explored by both papers allow for cheap and quick ways to get this testing done with some approximation of totality.

nathanyan 11:49, 15 October 2008 (UTC)

In general, ethnography seems like it can provide useful information about users' background and behavior that can't really be gleamed from contextual inquiry, since it delves deeper into other, less direct context of the user that while not directly applicable, can contribute to a greater understanding of a user's needs.

Through the ethnography methods described in the first reading, I'm surprised there isn't some sort of "pre-design" ethnography method. It would seem like this method would be most effective, to gain context about users *before* the design phase even began, to limit the amount of number of features or design models that are locked in.

Karen Tran 12:45, 15 October 2008 (UTC)

Ethnography is a type of social science research that investigates the practices and life of a community, by becoming one of its members. It is based on learning about a context and the people living in it, by understanding their values, needs and vocabulary. From the readings, it seems to me like it requires long periods of time playing this role and a faithful report of what is experienced or observed avoiding any interpretation or evaluation as far as possible. I think it’s very similar to contextual inquiry in the sense that they are both aiming for the same goal – trying to understand the users. However, contextual inquiry seems to be the quicker and easier way because besides the fact that it only requires interviews, instead of doing close observations of its users. I guess it is because of this close study of users that ethnography answer better the questions like what is necessary to innovate with success; what are the key social actors and roles to take into account; and which are the limiting factors?

Kai Lin Huang 15:20, 15 October 2008 (UTC)

The ethnographic methods of human-computer interaction research differ from contextual inquiry because of the researchers may act as a border between the end-user culture and the user interface. Many ethnographic methods are costly because of its long duration input according to David R. Millen in his research, and there have been research on how to condense the ethnographic research without losing much of the information that could be obtained in the lengthen research otherwise. I believe this process is becoming easier to achieve as technology and Internet advance to the future stage. However, ethnographic methods of research in human-computer interaction could have to change at the same time because more technology becomes available and, as the developers of those technology envision, those technology will change the way people work. Some traditional ethnographic research methods may not be suitable to many new situations anymore, such as the widely availability of web 2.0 applications.

Kevin Lam 15:52, 15 October 2008 (UTC)

It's interesting to learn that ethnography encompass the contextual inquiry method we learned about before. From what I gathered in the articles, ethnography is distinguished by its focus on cultural context. Assessing the culture of a workplace is important, in that it gives the surveyors an unbiased view of the user and the user's workplace. The culture of a workplace can affect the way an employee goes about his every day tasks. Rather than attributing an employee's behavior to his personality, we may want to see how the environment plays a role. It may be true that the employee is distanced from his work or not able to function at his full capacity, because of some limiting factor in the workplace. These factors can be unveiled with an ethnographic study.

In terms of the method itself, it seems that ethnography can be a highly effective surveying model but in specific cases. The time requirement and the scalability limitations make it harder for interviewers to draw broad conclusions about the workplace. Instead, it would seem as if you can only make specific assessments unless you were to spend weeks observing users.

Another thing that drew my attention was the rapid ethnography method. In today's fast paced, web 2.0 world rapid surveying is critical in order to keep pace with competition. At the time the Maryland article was written (2001), it may have been ok for an interviewer to spend weeks gathering information on a project but doing that today could give the competition enough time to develop a better product. Of course, this isn't true of all situations. In large scale systems implementations, it is more desirable to take the time to gather requirements and information about the users and the culture of the workplace before proceeding with the implementation. One reason for this is that a lot of systems implementations are contracted out, so there is little to no need to worry about competition. However, project time constraints may still come into play. As a project manager, you would want to complete your information gathering in a timely manner so you have sufficient time to implement and test the project before the deadline.

Yuta Morimoto 16:05, 15 October 2008 (UTC)

Ethnography seems to be a kind of procedures for observing a user. I think in the context of Ethnography I just observe users and record their behavior like the sociology or ecology. This procedure suggests just observing users so that I analyze them objectively. I think scientifically Ethnography is the best method to figure out user behavior. Because, it realizes the observation of task of user with out any prior knowledge user's behavior. However, as authors mentioned in both readings, Ethnography is too time consuming. I think it is applicable for developing HCI that "Third, use collaborative and computerized iterative data analyze methods." as Millen said. Since, I can easily collect statistical information of user behavior, if I implement such function before/after making the software. Although this method require me to make a prototype, I do not need more time to observe users.

Juanpadilla 16:12, 15 October 2008 (UTC)

The idea is really good though. I have worked for a few, very large, corporations, each with their own "culture" so to go in and get a feel for this and then provide software that fits in to the way things are done is a really good thing. On the other hand, software such as MS Excel would be extremely difficult to evaluate with this process since it has to cater to such a large, diverse user-set. For example how would you relate the cultures of an experienced Wall Street office worker creating a spread sheet for the ump-teenth time to a middle schooler just learning to create their data set for their science class. It may be possible, but highly difficult is all I'm saying.

Mike Kendall 16:24, 15 October 2008 (UTC)

The problem with ethnography has always been the conflict between the ethnographer's world and the subject's. is the ethnographer supposed to impose their value system on the subjects? This is the kind of incorrect study that leads to statements about how horrible employees are getting paid in other countries. Sure they're getting $7 a day, but that's great pay in the Philippines. The other extreme is if the ethnographer decides to participate and (although this is particularly difficult) becomes a member of the subject population.

In some cases it is a good idea to compare your culture and experiences to others', isn't it? In some cultures, being sold into a job for life is pretty normal, but we believe that this is too wrong or too different for us to let happen (case in point: M*A*S*H). This is all a reaction to the word "unbiased," because ethnography is inherently biased. There is almost nothing objective about field work. However, in some cases this is a good thing.

Stuart Bottom 16:45, 15 October 2008 (UTC)

It seems the greatest barrier to using ethnography "in the wild" (that is, for real HCI research) is time. A good ethnographic study may take months, and most HCI design groups don't have the kind of time or money needed to implement that sort of in-depth study. If time is the essential requirement for a good study, the real question is is it possible to gain the necessary information in a compressed time sequence? The second reading detailed methods for doing this, but I'm still not convinced that the accuracy for "rapid ethnographic methods" is nearly as high as a full-on drawn-out study. To me, the rapid methods are still too error-prone; for example, how can you be sure that you have found honest or in-the-know key informants? Also consider: the less time you spend with a person, the easier it is for them to give you a different impression of what is actually happening. People aren't necessarily dishonest, but they can be (for example, if they are trying to put on a good face and give a positive impression of their work environment to outsiders). It is easier to miss things when you are in a hurry. The fact remains that less time spent observing people in their work environment equals a higher chance of deriving incorrect conclusions from them - and HCI researchers would still do well to be wary of this danger.

Trinhvo 17:00, 15 October 2008 (UTC)

Ethnography is a way to understand where our users' background, what they expect, and what their view points are. People are from everywhere, the user interface should be designed depending on who they are designed for.

Paul Im 17:06, 15 October 2008 (UTC)

Ethnography seems like another ways to extract information from users, much like a contextual inquiry. There are different strategies you can use based on your own time and resource constraints. Some of the methods give you a time advantage and others give you a resource advantage. Another difference, however, is that there doesn’t seem to be anything like an apprentice model. The user, by the use of various methods, can relay information to us without being in context. Thus, with correct selection of users and methods, we can perform a more rapid analysis. Ethnography claims to have the same depth of analysis as any other inquiry method, but one can’t help but wonder if such rapid methods sacrifice some level details here and there.

Greg Nagel 17:15, 15 October 2008 (UTC)

I think Ethnography has a place in business, but not for a specific project. I'd like to see it treated more as employee training: teaching them about the company's main user base, gathering information for all projects as a whole. It's so easy to get wrapped up in your own head, thinking everyone's requirements are the same as yours. But you can't create a good user interface that way.

Jonathan Fong 17:20, 15 October 2008 (UTC)

I have a particular fondness for "soft" sciences such as anthropology and ethnography (though some wouldn't even consider these sciences at all!). My first experience with this was a discussion where I learned that financial companies frequently hire anthropologists to study start-ups that are about to be acquired by corporations. This is part of the "due diligence" in the Mergers and Acquisitions process. The corporation's goal is to acquire the start-up, but keep it profitable; forcing a corporate/company culture onto the startup frequently undermines this goal, so understanding company culture is very important.

In general, the study of people and how they behave in groups and in society is a valuable context into their preferences, values, and goals. Any study into how external factors will influence these people can be used to facilitate the change management process. Getting people to successfully adopt new practices is successful change management! Our serious game hopes to change an aspect of the user's life, so understanding the lifestyle of the user is an important tool in our development process.

Volodymyr Kalish 17:21, 15 October 2008 (UTC)

Internet not only made information more available to everyone, but also brought many kinds of users closer together. So, during the design phase of a product, especially an online application, it would be very beneficial to take ethnography into account. The better we know users and their background, the better we can satisfy their needs by predicting their behavior. So, ethnographic study of users can reveal a lot of useful information about the target user group.

Search engines probably use that information when displaying ads.

Cynthia T. Hsu 17:22, 15 October 2008 (UTC)

It was interesting that both these papers emphasized similar principles to what we've been learning in class, such as large scale user evaluations and using "interactive observation techniques" (such as lo-fi prototyping).

I thought it was interesting how the Millen paper described, "While often misconstrued as simply a method of field data collection, ethnography is rather a form of analytic reportage, with the ethnographer acting as a translator or cultural broker between the group or culture under study and the reader." Even though human cognition interface design is for the purpose of creating something usable by humans, this almost seems to suggest that the designers are a very specific type of human which may not be able to comprehend the group or culture under study. While this is probably true for Professor Canny's research about human computing interfaces in developing countries, it's hard (though plausible) to imagine it being true for a lot of software - it's easy to assume a "basic familiarity" with computers, although this term is variable.

MuQing Jing 17:22, 15 October 2008 (UTC)

From the readings, I get the feeling that ethnography is a much looser and less strict version of gathering information used for design. It's similar to contextual analysis in what it hopes to accomplish, but in a much broader scope. While contextual analysis focuses on observing the users directly in a very specific environment, ethnography looks at culture in order to make generalizations as to what the user wants. It is interesting to note that in ethnography, unlike in contextual analysis, tasks aren't directly observed and recorded; rather, it takes an expanded view and doesn't bog itself down with specifics. This makes it seem like ethnography is able to more effectively cover more ground in the design process, but isn't as suited as prototyping for the actually specific tasks that users are required to perform.

Geoffrey Lee 17:25, 15 October 2008 (UTC)

One interesting point I found in the article about Ethnographic Methods is that observing the user in his/her natural setting can reveal "unspoken" needs. It seems as if the user is unaware of their own needs, or perhaps they only feel the need for things that they are consciously aware of. I feel that this gives Ethnography a huge advantage over simple interviews of users.

However, one distinct disadvantage of Ethnography is the amount of time investment required. It clearly takes much more time to examine 50 users using ethnographic methods than it does using a mail-in interview. If too few users are studied, the results will most likely be statistically inaccurate, so this is definitely not something that should be used if the examiner isn't willing to put in significant time investment.

Shyam Vijayakumar 17:29, 15 October 2008 (UTC)

Ethnography is similar to contextual inquiry, except that it can be done faster and with fewer resources. These differences have both advantages and disadvantages. Ofcourse, less time and resources spent on the observational part of the process means that you probably will have more problems later on with the high fidelity prototype perhaps. At that point, the developers will not have the freedom to make too many changes. But if something needs to get done quickly and perhaps the developers do not have too many resources to begin with, rapid ethnography seems like a good fit. This model would have been perfect for me, when I had to develop a prototype over the summer. I obviously did not have a lot of time and resources. Also, the purpose of my prototype was to specifically deal with an employee's culture of various enterprise-related applications that he/she uses. All these properties seem like a perfect fit for rapid ethnography. I really wish I took this class before the summer...

Retrieved from "http://bid.berkeley.edu/cs160-fall08/index.php/Qualitative_Evaluation"
Personal tools