Seminar: Games User Research

In this seminar you will plan and conduct a user study in the context of games and game culture (and, of course, play a lot!). We have 16 spots in this seminar. In groups of four (the groups are formed in the kick-off), participants will - based on the related research in this area - decide on the research question they want to explore and develop a research methodology on how to investigate this question. The range of possibilities in "Games User Research" is endless, some examples:

  • You can do research on available games (e.g., [1-3, 10-13, 17], where specific aspects of Portal 1, Left4Dead/Rocket League, Pinball/Tetris/THYFTHYF were investigated) or research on specific aspects games (e.g., NPCs [19] or receiving random rewards [20]).
  • You are also free to develop “mods” or “extensions” for existing games to introduce new stimuli and to explore the effect of these (e.g., [4], where new difficulty options in a Tetris game were introduced and evaluated).
  • You can do studies related to the game culture, e.g., how a particular game is played on live-streaming platforms or how streamers can be better supported by their audience in this game (e.g., [7], where Hearthstone streams were investigated), or how the effect of music influences board game players [18].
  • You can develop a game concept, prototype, and playtest it (e.g., [5, 6]), or investigate effects other players have on the game play/performance of a player (e.g. [15, 16])
  • You can explore the experiences players have with a particular game (e.g., [8], where the Dark Souls III experiences of players were analyzed) or with games in general (e.g., [9], where the emotional components of games were analyzed).

Participants are free to select whether they want to consider a commercially available digital game, an open-source game, an analogous board or card game or even an activity-based game (e.g., hide and seek; playing with nerf-guns, ...). The only requirement is that they already have access to it and all involved parts (e.g., we are unable to buy you weapon skins if you want to explore the effect of such customization). It is also ok, if no specific game is focus, but a survey (or similar) on specific aspects of game is targeted. As long as you provide a solid research question in the games domain there is a high chance that it fits the scope.

We gave this seminar in 2018 and to give you just one example one study done had the goal to investigate the effects of listening to music while playing Tetris.

To give you a little incentive to read the whole page and for us to see whether you have read the whole page, you need to play a little game: you need to provide a solution to the motivation field in the CS seminar system. It consists of five letters that you will generate by solving riddles here on the page and that you need to arrange to a word. In addition to the correct word, you also need to provide a brief motivation, why you think this seminar is a good fit.

Important Facts & Grading

  • Because of the above-mentioned flexibility this seminar will last the whole semester (including the lecture free time). Only register for this seminar when you are available in the lecture free time as well: See the dates section below. But also read the information here on the page to see the workload distribution. Only register if this is acceptable for you. Take the first letter of the antonym of “the beginning”.
  • Because of the late run of the seminar assignment constraint solver and internal matters, the kickoff will only happen in the third week of the semester. As all relevant information to the seminar can be read on this page, it will be mainly a Q&A session.
  • To pass the seminar you must …
    • … attend all seminar meetings (including the Kickoff!) AND
    • … you must take part in all user studies of the other groups AND
    • … pass all grading parts (i.e., at least achieve a 4.0).
    If one of these criteria is not met, you will fail the seminar (i.e., get a 5.0).
  • Take the 18th letter of the Latin alphabet.
  • It is not necessary to implement something in this seminar, but you should understand how to conduct user studies and how to analyze results. This is the reason we require you to have successfully participated in the Human-Computer Interaction lecture. In addition, depending on your research question, having passed the Statistics with R lecture is beneficial as well. In the seminar there will not be time to explain general aspects on how to conduct user studies or how to analyze study results. We also expect participants to acquire further knowledge in parallel.
  • Grading: 20% GUR Paper presentation; 60% Study preparation, execution, and analysis; 20% Final presentation (and written report)

Seminar Structure

Phase I: Introduction to Games User Research

Every student receives one scientific paper from us, directly after the assignment to the seminar. This paper should be read and presented individually in a 12-minutes presentation. Here, every student should present the research question of the paper, which games were used, how the experiments looked like, which data were collected, how it was analyzed, and which results were made. The introduction has the purpose to give all participants an overview of how Games User Research is done today and how possible experiments can look like. This presentation counts 20% of the grade and we will make your presentation accessible to all participants. This seminar is given in which term? Use the fourth character. In parallel, you should start reading the Games User Research book (accessible via Uni-VPN) AND ensure that you have a solid understanding of “qualitative” and “quantitative” research methods and data analysis.

Phase II: Preparation of the Study

After an initial pitch of several ideas with the advisor, the group together with the advisor focus on one idea only. For this idea, the group develops and writes down a research concept describing which research questions should be investigated (take care that these have not already been investigated in the literature for this game or game genre: i.e., do a scientific literature check!). This concept should cover the game that is used (if any and you can use different games in your study, if necessary), the method of the study (i.e., the procedure and which data you want to collect and how) and how you will analyze the collected data (both should be explained as detailed as possible). We will have two writtenn feedback cycles for this phase. What looks like a typo isn’t one. The groups have time to set up everything that they have planned in their concept.

Phase III: Conduct the Study

Before doing the study with external participants, you will have a test-run with your group’s advisor. Only after he gives you a “go”, you can start with the study and other participants. Every participant is requested to participate in every other study. In the best case (where everyone can participate in every experiment and with a fully booked seminar) this means that every experiment can be conducted with at least 12 participants without the need to find other participants (if you want or if this is necessary for your research question, you can/should also recruit further participants). Following your study concept, you will analyze the acquired data. Formulating research questions, the study concept, the execution of the study and the analysis of the study data counts 60% of the grade.

Final Presentation

In a 30 minutes long presentation (+ 15 minutes discussion) the groups will present their research questions, why these are interesting from a scientific point of view (i.e., you should present relevant related work), the study concept and the results. Convince the others that you have successfully conducted a "Games User Research" study! This presentation serves as a final report for this seminar, however, if you want, you can also submit a document that illustrates all your findings, in case the presentation time was only sufficient to present the main findings. This presentation counts 20% of the grade; if you submit a document, we will consider this for grading as well.

Dates

Our seminar slot is on Thursday, 12-2pm (only the final presentations will be 12-4pm). We will meet in room TBD. For the solution, use one of your findings twice.

  • 31.10.2022 – Assigning papers to students via email. Papers can be accessed via Uni-VPN. Please start to prepare a presentation (hand-in 16.11).
  • 10.11.2022 – MEETING: Kickoff (Q&A-session and group assignment).
  • 16.11.2022 – DEADLINE: Hand-in of presentation material (all students, independent on their presentation date). Groups should start to brainstorm study ideas.
  • 17.11.2022 – MEETING: Presentation of GUR Papers (6 papers)
  • 24.11.2022 – MEETING: Presentation of GUR Papers (5 papers)
  • Until 25.11.2022 – DEADLINE: Ensure to have had at least one meeting with the group’s advisor, where you have pitched study ideas (a pitch should highlight the research question, the game or game-related aspect and how a study would look like). As preparation of this meeting prepare a brief document (max 2 pages) with the ideas you will pitch. With the feedback after the meeting, you should focus on just one idea in the following.
  • 01.12.2022 – MEETING: Presentation of GUR Papers (5 papers)
  • 08.12.2022 – DEADLINE: Hand-in the research concept. You will receive feedback until 09.12. If you plan to implement something, it is reasonable to start as soon as the feedback indicates that the implementation will remain in the concept.
  • 06.01.2023 – DEADLINE: Hand-in the refined research concept. You will receive feedback until 09.01.20223. Now you have time to prepare everything. Also, together with the advisor, contact the Ethical Review Board.
  • Until 17.02.2023 – DEADLINE: Set up everything and ensure you had a test-run with your group’s advisor before this deadline. For this, assume that the advisor is a participant who has never read anything from your study setup and treat him as you would treat external participants. Expect changes and ensure to have enough time to implement/do these for the actual data collection.
  • Between 01.03.2023 – 10.03.2023 – Main data collection phase. Run the study. All students must take part in experiments of all the other groups. In addition, more participants might be needed depending on the research question. In such a case these additional participants need to be recruited by the groups.
  • Between 13.03.2023 – 29.03.2023 – Do the data analysis, prepare the final presentation, and prepare the report.
  • 30.03.2023 – MEETING: Final presentation (12-4pm).
  • 30.03.2023 – DEADLINE: Hand-in of the written report and all study data.

Organization

Dr. Pascal Lessel, Dr. Maximilian Altmeyer, Marc Schubhan

Literature Examples

  • [1] Visani Scozzi, Monica; Iacovides, Ioanna and Linehan, Conor (2017). A Mixed Method Approach for Evaluating and Improving the Design of Learning in Puzzle Games. In: The ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play (CHIPLAY 2017), ACM, 217–228.
  • [2] Kellie Vella, Christopher James Koren, and Daniel Johnson. 2017. The Impact of Agency and Familiarity in Cooperative Multiplayer Games. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '17). ACM, 423-434.
  • [3] E. Kimble, Charles & S. Rezabek, Jeffery. (1992). Playing Games Before an Audience: Social Facilitation or Choking. Social Behavior and Personality: An International Journal. 20. 115-120.
  • [4] Dennis Ang and Alex Mitchell. 2017. Comparing Effects of Dynamic Difficulty Adjustment Systems on Video Game Experience. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '17). ACM, 317-327.
  • [5] Jason T. Bowey and Regan L. Mandryk. 2017. Those are not the Stories you are Looking For: Using Text Prototypes to Evaluate Game Narratives Early. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '17). ACM, 265-276.
  • [6] Joe Marshall, Conor Linehan, and Adrian Hazzard. 2016. Designing Brutal Multiplayer Video Games. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, 2669-2680.
  • [7] Pascal Lessel, Alexander Vielhauer, and Antonio Krüger. 2017. Expanding Video Game Live-Streams with Enhanced Communication Channels: A Case Study. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, 1571-1576.
  • [8] Serge Petralito, Florian Brühlmann, Glena Iten, Elisa D. Mekler, and Klaus Opwis. 2017. A Good Reason to Die: How Avatar Death and High Challenges Enable Positive Experiences. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, 5087-5097.
  • [9] Julia Ayumi Bopp, Elisa D. Mekler, and Klaus Opwis. 2016. Negative Emotion, Positive Experience?: Emotionally Moving Moments in Digital Games. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, 2996-3006.
  • [10] Rina R. Wehbe, Edward Lank, and Lennart E. Nacke. 2017. Left Them 4 Dead: Perception of Humans versus Non-Player Character Teammates in Cooperative Gameplay. In Proceedings of the 2017 Conference on Designing Interactive Systems (DIS '17). ACM, 403-415.
  • [11] Madison Klarkowski, Daniel Johnson, Peta Wyeth, Mitchell McEwan, Cody Phillips, and Simon Smith. 2016. Operationalising and Evaluating Sub-Optimal and Optimal Play Experiences through Challenge-Skill Manipulation. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, 5583-5594.
  • [12] Jan D. Smeddinck, Regan L. Mandryk, Max V. Birk, Kathrin M. Gerling, Dietrich Barsilowski, and Rainer Malaka. 2016. How to Present Game Difficulty Choices?: Exploring the Impact on Player Experience. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, 5595-5607.
  • [13] Pascal Lessel, Alexander Vielhauer, and Antonio Krüger. 2017. CrowdChess: A System to Investigate Shared Game Control in Live-Streams. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '17). ACM, 389-400.
  • [14] Emmerich, K., & Masuch, M. (2013). Helping Friends or Fighting Foes: The Influence of Collaboration and Competition on Player Experience. In FDG, 150-157.
  • [15] Katharina Emmerich and Maic Masuch. 2017. The Impact of Game Patterns on Player Experience and Social Interaction in Co-Located Multiplayer Games. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '17). ACM, 411-422.
  • [16] Ansgar E. Depping and Regan L. Mandryk. 2017. Cooperation and Interdependence: How Multiplayer Games Increase Social Closeness. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '17). ACM, 449-461.
  • [17] Karina Arrambide, John Yoon, Cayley MacArthur, Katja Rogers, Alessandra Luz, and Lennart E. Nacke. 2022. “I Don’t Want To Shoot The Android”: Players Translate Real-Life Moral Intuitions to In-Game Decisions in Detroit: Become Human. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). ACM, 1–15.
  • [18] Timea Farkas, Alena Denisova, Sarah Wiseman, and Rebecca Fiebrink. 2022. The Effects of a Soundtrack on Board Game Player Experience. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). ACM, 1–13.
  • [19] Martin Johannes Dechant, Robin Welsch, Julian Frommel, and Regan L Mandryk. 2022. (Don’t) Stand By Me: How Trait Psychopathy and NPC Emotion Influence Player Perceptions, Verbal Responses, and Movement Behaviours in a Gaming Task. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). ACM, 1–17.
  • [20] Michael Yin and Robert Xiao. 2022. The Reward for Luck: Understanding the Effect of Random Reward Mechanisms in Video Games on Player Experience. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). ACM, 1–14.