Group Low-Fidelity Prototype:Group Phi-tus
From CS 160 Fall 2008
Contents |
Each team member’s name and role in this assignment
- Kevin Friedheim - Intro/Mission Statement, Procedure, Discussion, Test Measures, participant in testing process + prototype sketches.
- Frank Yang - Results, Discussion, testing process and prototype sketches.
- Xuexin Sean Zhang - Participant, Prototype Description + Prototype Sketches
- Haosi (Jason) Chen - Participants, Environment, Discussion, prototype sketches, and gathering interview notes.
- Juan Padilla - Taks, participant in testing process and prototype sketches.
Introduction and mission statement
We are implementing a game design that requires participants to converse in the language they are trying to learn a foreign language. Our interface currently is very generic, and therefore flexible and so we plan on modeling our user interface based on interviewee's feedback. The purpose behind the project is to determine if by mixing fun with learning, the product of such game-play is an edification that could not be obtained by studying alone in the same time frame. By placing users on different teams within the game, a competitive edge is given which drives users to not only draw upon existing knowledge of the language that is being learned, but it also drives users to build on this knowledge.
Mission Statement: Our mission is to facilitate for our target users (individuals trying to learn or otherwise master a foreign language) the process of learning by fusing this process into a fun game. We plan to do this by using our version of a blend of Pictionary®/Taboo® types of games -- as it would apply to learning a language. Our research has shown that the vast majority of the language-learning process involves verbal communication in the language in question. Our goal in this design is to bring this "best" aspect of learning into an environment where participants will try very hard to both speak and comprehend the language.
Prototype description, with sketches and a picture of the entire system
In our Lo-Fi prototype, we loosely defined user's experience in term of Human-Computer Interaction. Our Lo-Fi prototype of the serious game contains the user interfaces of seven different stages of the game.
The first stage of the game is the Home Screen of our serious game that contains the elements including a text input area for user to enter his or her Nickname during the game, a button to create a new game, a list of existing games, and also a help button available on the top right of the Home Screen. In the list of existing games, the lock icon indicates that's a private game which requires a password to join; games are listed by their language, level of difficulity, and the number of current player / maxium player. A "Join" botton follows the game desciption. Once clicked on the Help button, a pop up help menu will be displayed and gives players a more detailed instruction for the Home Screen.
After clicked the Create a New Game button, players will be presented the Create Game Screen, where the creater get to select Language, Level of difficulity, a minium number of player to start the game, as well as a password if the creater want to create a private game. Beside those, the player will be click the "Create" to create the game or click on the "Back" button to go back to the Home Screen.
The thid stage is the Waitting Room Screen in which players wait for other players to join the game while chatting with other players already joined. At the top half of the Waitting Room Screen, there are slots of roles of Dictator and Drawers. In each slot, it contains the avertar of the player along with the statics data from his or her past game plays. If there is a slot open, a player could choose to click that opening slot to switch to the role of that slot. On the bottom half of the Waitting Room Screen, there is a chat box with text input bar for players to chat while waiting, and there is a "Cancel" button to exit the game, and a "Start" button to start the game once there are enough players.
In the Game Starting Screen, the player will be confirmed with their role while getting ready to play with a five second count down.
Once the game begins, players with the role of dictator will see the Dictator Screen while the drawers will see the Draw Screen. On top of both screens, it has player's current score, Remaining timer for dictating/drawing, along with a help button and a back to home button. In the Dictator Screen, player will be given a picture of an object, and a list of Taboo words that are forbidden during the game play. At the lower right corner, there is a Talk Icon lights up when then dictator speak to the drawer and also a volume controller to adjust the volume.
In the Drawer Screen, there is a white space in the center for the drawer to draw down what the dictator describes using the tools given on the toolbar at the left-hand side of the screen. The toolbar consists of a selector tool, a pen tool, a straight line tool, a filler tool, a easaer tool, a color selector tool, and a shape tool. At the buttom of the Screen, there is a clear button for the drawer to clear the content of the drawing space and a submit button to click after finish drawing.
After every drawer finished drawing or the time is up, each dictator will be grading every anonymous drawing in a random order. In the Dictator Rating Screen, the dictators will be presented all the drawings and below each drawing, there is a star bar where dictarors could rank those drawings. Once finished rating, dicator could click on the submit button to commit the result. On the button part of this screen, there is a chat box where dictators could talk about what they think and what they have learned.
While the dictators are voting on the drawings, drawers will be given the orginial picture that dictators saw, and they could see the real time result of the voting. On the top of the screen, there are drawings from different drawers along with their real time score. Below that is the orginal picture to refer to. On the button, there is also a chat box where drawers could exchange ideas.
After the rating, everyone in the game will see the Winner Screen which contains the winning drawing and the nickname of its drawer and dictator. And the game will restart to Waitting Room Screen with new ramdomized pairs of drawers and dictators.
Method
Participants
Since there is player amount requirement for our game, our group agreed on interviewing four interviewees at once for our Low-Fidelity Assignment. We also try to find people that with different major backgrounds. We found Chriss (3rd year EECS student) and Kim (3rd year IEOR student) randomly outside of Soda Hall when they were talking about their class project for a course. Then we went to south side, sather gate area for interviewee hunting and found Amy (3rd year econ student) and John (4th year biology student). After we gather all the interviewees we need, we performed our prototype testing with them in Soda Hall.
Environment
We arranged our testing environment to perform a group testing. Since our game requires multiple players to be the dictator and drawer , we have interviewed 4 people at once for the testing. We perform our test on a huge table that we were able to let the dictator sit on one side of the table and the their corresponding drawer sit across them. The "computer" sat at the end of the table. The facilitator was seated on the left of the user, with a clear view of the user/computer/prototype interactions. The table was large enough so that we could lay out the pieces of the prototype in front of the "computer" even when they weren't in use. The user initially have the "home screen" in front of them, and as the testing process go, the "computer" would pass the specific "screen" to the players. The user interactions with the application were all done by touching elements on the "screen".
Tasks
After carefully considering the list of tasks available to the users, we chose the three tasks we felt were the most representative to our game that also covered the major components of the user interface. The three tasks we chose were creating a game as the task, rating the drawings as the task and drawing a picture as the task.
Rate the drawings (easy task): We felt that rating a drawing was a relatively easy task. The user is simply presented with a screen showing all of the drawings that have been submitted from the last round with a set of an unfilled stars used to perform the rating under each drawing. The rating process is similar to that of rating a product on Amazon.com, Apple.com and various other familiar websites. As such, the user must simply hover the input device over the stars and once the amount of the stars is “filled”, the user then clicks to lock in the result. The user repeats this process over the number of drawings then selects the submit button for the results to be tallied.
Create a new game (moderate task): Game creation was chosen as our moderate task because it requires the user to interact with elements not used in other parts of the game; namely, the information form that is presented to users in order to choose the basic parameters of the game such as language and difficulty level. This form is reached by selecting the “create a new game” button form the main screen. Once here, the user chooses the game’s parameters, which are mainly guided through a set of drop down menus for easy setup. Once the user is satisfied with their choices they can click the “create” button and are then taken to the waiting room where they await other player to join their newly created game.
Draw a picture (difficult task): Drawing a picture is one of the fundamental tasks of the game so we felt it very desirable to make sure that we got this right. The screen we used is very simple and familiar to that of most drawing programs such as Photoshop but not nearly as robust. The task simply requires the user to select the, mostly familiar, tools in the left hand toolbar such as the shape or pencil tool and use them to draw a picture. The task of selecting the proper tools to create the desired effects was the crucial step for this task. Once the image is drawn, the user can either wait until the timer runs out or submit the picture early for possible bonus points.
Procedure
Describe some details of your testing procedure. This should include the roles of each member of the team.
Due to some availability problems, our whole team was not able to participate in every testing procedure -- but we were still able to have at least a greeter/facilitator, a computer simulator, and an observer, which worked out perfectly.
Greeter: Our greeter/facilitator had the responsibility of ensuring the comfort of our participant and ensured that the atmosphere never became too awkward for the participant. The other purpose of our greeter was to help with getting data from the participant.
Computer: Our Computer had the responsibility of simulating the prototype by response to the participants "mouse click." He was very careful not to help or guide the participant.
Observer: The Greeter also participated as the observer and helped take down notes and observations, as well as recommendations granted by the participant.
Prior to participation, all demonstration materials should be in place and ready so as to facilitate participation and so that the testing procedure occurs in a timely fashion. We went though the following routine -- stopping to ask questions during different phases of the testing procedure. (note, this is only a brief summery of steps, see attachments for further description).
-The participant is welcomed and greeted and made as comfortable as possible. -After informal introductions, begin with presenting the consent form and explain that it is only a requirement of the assignment and that s/he should read it over, sign, and date the form. -After this, the participant is informed about what our game is about, how it is played (simple rules/etc...). -Next explain the how participation should occur -- in other words, explain that suggestions are welcomed and encouraged, thinking out loud, exaggeration of any difficulties with the interface. -Now begin the tasks -- [task 1: rate drawings | task 2: create a new game | task 3: draw given a description in foreign language]
-Task 1: You will be performing the task of rating drawings that "drawers" have drawn for their perspective "dictators." Here you will judge/rate the drawings based on how accurately they match the drawing that was given to the dictator at the beginning of the round. -Task 2: You want to create a new game, instead of joining a game. Proceed though the interface to accomplish this and tell us when you think you are done. -Task 3: In this task you will be required to sketch, as best you can, the person/place/object/etc that will be described by the dictator. Do so as best you can given the time constraint at the top of the "window." The "computer" will let you know when you have 10 seconds remaining - at which point you should do your best to decide what is being described and complete your sketch. -At this point a list of generic questions/any other questions should be asked. -Participant should be thanked and dismissed. -Team should fill in any gaps in the note taking and all should collaborate and offer any observations witnessed.
Test Measures
TThe test measures detail what you looked for or measured during the experiment. You should concentrate on process data (i.e., what is happening in the big picture) in addition to bottom-line data (i.e., time or number of errors).
Our biggest concern was whether or not certain aspects of our game were straight forward or required further explanation. It was these very "aspects" that helped guide us in choosing which tasks we would have our participants perform. The follow is a brief recap of each of the three tasks along with the test measures that we had in mind for each of those tasks:
The first task that we had our participants perform was the easiest one (or at least one that our group was able to deem "easy") - judging from best/favorite from a set of drawn pictures, and then placing a rating for each one. Our goal was to test to see if the user was able to understand how to use the rating system (the stars). When coming up with a rating system, there was some debate as to subtle differences in "star-like" rating systems. That is, some are right-to-left rated, and others are left-to-right. Rating is a huge, if not the biggest, part of the game as it represents the reason that the game is even played (aside from the learning aspect).
The second, moderately difficult task involved the participant going though the process of creating a game (as opposed to joining a preexisting one). Although this was not our most "difficult" task, we feared that it might prove to be so for a user that was not familiar with our design scheme. The reason for this fear is that our placement of the button was difficult/awkward and we were afraid that it might not be clear what to do and what to do next from screen to screen. Also, it was hard to represent a beveled button on the paper low fidelity prototype, so a button (which was what we used to represent the "create game" icon) might be easy to miss and/or easy to overlook. Therefore, the consensus was that we would test for this particular problem by creating a task out of it.
Finally, the final, most difficult task was to draw. Now normally you might think that drawing is an intuitive activity that anyone who has ever used a computer has performed over and over again. But tools that users utilize on different platforms using a variety of programs vary greatly in some cases, and subtly in others. The problem, and one that we felt merited testing within a task, is that those subtle differences could confuse the user; and, in a timed game like ours, confusion can be the difference between winning and losing, and more than that, the difference between playing our game and not.
It should be noted that prior to, during, and after each of the before-mentioned tasks, the participant was questioned as we saw fit. Although in some cases, even though the participant appeared to have no or little difficulty with a task (i.e. selecting the correct tool to use when drawing), we still asked just to be sure.
Results
Summarize the results of the experiment from your data. The initial reaction of our participants was positive. All were intrigued by what the game had to offer. As soon as we presented our intervieweees with our low fi prototype, they were eager to get started. As soon as the interface was placed in front of our interviewees, each interviewee scanned the screen intently. Although they were already told their task, both Chriss and John began to ask us the question "How am I supposed to..." where we immediately told them that the objective of the experiment was for them to simply interact with the interface directly. All of our interviewees decided to use the help button provided at the top of the screen. After reading it, all interviewees attempted to hover over the stars to see if they could get an idea of how the stars would rank the pictures. When nothing happened, they tried clicking, and they seemed moderately confused as to why there were only three stars that could be selected. Kim mentioned at this point that she would probably turn to the chatbox and ask the other players for an answer to her rating procedure questions. Even as the interviewees clicked, Kim still did not know if the rating system was a ranking system or a just a normal rating system you would find on an application such as Youtube. Most interviewees discovered that their votes were not final until the submit button was clicked, but some attempted to clear the rating they had given.
Upon being given the second task, most of our interviewees directly clicked the Create A New Game button that was on the screen. However, Chriss mentioned that she would try to fill out the nickname first. She was then confused as to where she could submit her nickname to confirm that she could use it. She mentioned that she did not want to go through all the trouble of creating a game and filling out the menu only to discover that her nickname could not be used. In the create a new game menu, many just filled out most of the form with ease. However, when each interviewee reached the max player field of the form, all of our interviewees asked what the limits were. All the interviewees simply entered a random number, ranging from 3 to 50 players. All of the interviewees also selected English as the language, despite the introduction of this game as a way to practice a foreign language. Upon submitting the form to create a new game, all of our interviewees were shown the waiting room. While one interviewee figured out what the screen was, most of our interviewees were immediately puzzled by the screen and asked what the screen was. The puzzled interviewees were unsure if they had successfully created the room or if they were currently playing the game already. Kim immediately said she would go to the chatbox to ask her questions about the screen. She continued to ask what the labels of dictator and drawer depicted on the screen were. John asked if there was any way to identify the room that he had just created. He mentioned that perhaps he could guide his friend to the same room to play together but there was no way to identify the room he was currently in. Chriss also commented how there was no way to kick a person out of the room.
With the third task, all the interviewees decided to click on the help button first. After reading the help tips, all commented that the ruler icon was very misleading. While some ignored it, many commented they had no idea what the selector cursor would do. Upon selecting it, some tried to select items by clicking on lines directly, and some tried to select by clicking and dragging, trying to create a rectangle to select objects. <insert name> commented that an item like the selector is probably unnecessary. All were satisfied with the creation of shapes until they reached the freeform polygon tool. While none were surprised by how the tool functioned, Amy commented how the creation of the shape was inherently different than the other shapes, which required a click-and-drag approach to create the shape. While she was satisfied with how the tool functioned, she commented how the tool didn't seem like it belonged in that menu, and could easily be in the main toolbar itself. John commented how it bothered him how there was no way to tell how large of a brush he was painting with. Kim mentioned how there was no way to mute the speaker, a function that "could be handy." Most of our interviewees successfully "drew" a picture and submitted it by clicking submit. However, Amy was confused about the submit button, mentioning that she would expect the game to automatically submit the picture she has drawn when the timer ran out. She continued by saying that if the program would submit it automatically, there would be no use of a submit button, seeing how each round is only a minute long.
Discussion
Discuss your results. What did you learn from the experiment? How will the results change the design of your interface? Was there anything that the experiment could not reveal?
We discovered that our prototype is exactly that, a prototype. It needs a lot of work based on the critiques and insight that we gained from performing the lo-fi experiment. The critiques (listed below) are really only the beginning of the changes we plan to make to our design since many of the suggestions allude to other recognizable improvements that can be made. For example, Chriss said that a loading progress bar would give users an idea that they're waiting -- immediately (well almost immediately) we thought of the idea of having a status indicator to let the drawer know when the dictator is speaking, and possibly another status indicator to indicate to the dictator that a drawing action has occurred. This way either of them can figure out that something might be wrong (like one of the players went away from keyboard or 'afk,' or worse, a player has left the game and that notification was missed by one of either the drawers or dictator -- it's not fun playing a multiplayer game alone). This was just one example of the "appifinaies" that we got from the experiment, and, to be honest, we didn't expect to get as much as we did out of it as we had (I suppose that is what most designers believe).
It was clear from the interview that our rating system needed some clarification. Our current idea had unfilled stars on the rating screen without an obvious method for voting. While our original idea was to rate each drawing individually on its own scale, we poorly executed that idea itself by only allowing 3 stars to vote. After discussing, we decided the easiest change would be to add more stars for the rating stage and allow the drawers to see an updated average score of their drawing for that round.
In regards to the second task, to address the nickname submitting issues that Chriss brought up, we decided to create a login/home screen to the game. This would allow the players to create nicknames before entering the game to ensure that their nicknames would not be conflict with another player's. Since technically there could be no limit to the size of the games, it would be wise to limit the size of a maximum of 8. This would reduce the waiting time for a player waiting for the game to stop. Since the field to enter the value is currently a text box, we decided that it would be best to change it to a dropdown menu in order to make the size limits of the game clear to the game creator. Also, there was a general consensus that the waiting room was a confusing screen to see. When we had created the game, we had just assumed that the typical user would automatically know that the screen after a game creating screen would be a waiting room. However, this was clearly not the case, and we realized we needed to make it apparent that this was the waiting room for the game the player had just created. Furthermore, we decided that we should add some privileges to the game creator. Also, due to miscommunication, we also found the removal of the initial labels of dictator and drawer imperative, seeing how roles will change in between rounds.
As mentioned above, we learned the value of performing a low fidelity prototype versus a high fidelity one; and, more importantly we learned how useful user import (or any outside input for that matter) can be to a prototype. We learned how easy to design something and think that it is flawless, because you've designed it and nothing can possibly be wrong with it. Its when you get outside perspective that you find how wrong you were.
Looking through the feedback and by our own observation, it shows that for the most part of our design is on the right track, with the exception of a few cosmetic changes regarding visual feedback, and some additional feature for both the speaker and drawer. Most problems appear in the drawer scenario. Some important features such as the brush size and eraser size left out in our first design. During the interview, most our interviewees have pointed out this inconvenient. Therefore, we have done some changes to our prototype in order to add the missing features.
The low-fidelity interview process provided much needed user feedback for our next iteration; however this information is likely to only cover a subset of user issues we will encounter. This is precisely why user interface design is an iterative process, because in a real-world scenario a product will continually be refined with each pass; addressing interface issues as they arise.
