Usability Testing for Virtual Reality: Theory & Practice


Marco Speicher, Dr. Markus Weber (Centigrade)

The seminar will be in English!


Time / Location

  • 14:00 - 16:00, DFKI Main Building, Room Reuse


Important Dates

Date Phase Topic Material
18.10.2018   Kick-Off [pdf]
25.10.2018   Lecture: "Make presentations great again!" (Speicher) [pdf1] [pdf2
07.11.2018 I Related Work: Paper Presentations [zip]
15.11.2018 I Related Work: Paper Presentations [zip]
22.11.2018 II Lecture: "How to conduct & report a VR study" (Speicher)
Lecture: "Usability Testing" (Weber)
[pdf1] [pdf2
06.12.2018 II Lecture: "VR Evaluation Framework" (Speicher)
Lecture: "Evaluation Metrics in Practice" (Weber)
[games] [pdf1] [pdf2
13.12.2018 II Study Concept: Presentations & Report Submission Deadline  
10.01.2019   cancelled  
17.01.2019 III Implementation: Q&A (what do you need? how do you get it?)   
24.01.2019   cancelled  
31.01.2019 III Implementation: Demo Presentations & Deadline  
14.02.2019 IV Experiments: Q&A (Open Planning Session) & Material from 22.11.  
21.02.2019 IV Lecture: "How to analyze and discuss study results" (Speicher & Weber) [pdf1
28.02.2019   cancelled  
07.03.2019   cancelled  
14.03.2019 V Results: Presentation & Final Report Deadline (23:59)  




Phase 1: Related Work

Each group receives various "VR User Research" papers. These should be read, summarized and presented in a 7 minute presentation per paper. Here the student should (1) present the research questions of the presentations, (2) which games/applications were used, (3) how the experiments looked like, (4) on what and how they were tested and (5) which results were achieved. The introduction is intended to give all participants an overview of how VR User Research is conducted today and what possible experiments might look like. This presentation counts for 15% of the grade.

Possible topic blocks for face-to-face events

  • Usability testing in theory and practice
  • Presentations of existing studies: research questions, study design, results, etc.
  • Classical structure of a "study description", i.e. hypotheses, respondents, design, task, procedure, etc.
  • Possible evaluation metrics (3DUI/VR or general)
  • Presentation and discussion of own studies

Phase 2: Study concept

As a result, each group develops and writes a study concept that describes which research questions should be examined in its own study. This concept should include the game(s) or application(s) used, the method of the study and how the results of the study will be analysed. Each group will submit their study concept as written report and present their study concept in a 7 minute presentation, including  (1) the research questions of their study concept, (2) which games/applications could be used, (3) how the experiments will look like, (4) on what and how they will be tested and (5) which results are expected. This presentation and report count for 20% of the grade.

Potentially Interesting Aspects - Before your Own Study

  • In which "error classes" can usability problems be classified? (e.g. feedback problems, presentation problems, (low level) technical problems)

Phase 3: Implementation and Study Prototype Presentation


Phase 4: Conducting Experiment

According to the concept, the experiment should be carried out by the group. Each seminar participant will be asked to participate in any other study (if not, you will not receive a certificate for this course), i.e. at least X participants per study.

Potentially Interesting Aspects - After the Own Study

  • Which usability problems were to be expected?
  • Which usability problems were surprising?
  • What was simple/difficult in the organization, execution and evaluation of your own study?
  • What would you do differently in the next study?

Phase 5: Analyzing & Presentation of results

After the analysis of the results, a 20-minute presentation (15 min presentation + 5 min discussion) will take place. The groups will present their project together with related papers, study concept, study description, as well as the results. This presentation serves as a final report for this seminar; if required, a written elaboration can also be submitted, which only has a positive effect on the grade, e.g. to improve the grade.

Questions about the event concept

  • How are the "test objects" for the individual groups selected/assigned?
  • Which infrastructure can the participants access for the tests?
  • In what time period should the studies take place?
  • How is this interlinked with the attendance dates? (For example: Is there a certain period of time in which the studies are carried out by the participants and in which there are weekly seminar dates in which "study-independent" things are discussed/presented?)



Registration by E-Mail is not possible! 

Please use the following application form for registration: (closed)


Prior knowledge

User Interface Design / Human-Computer Interaction
Grundlagen der Medieninformatik
Einführung in 3D User Interfaces / Virtual Reality
The seminar is aimed at Bachelor/Master Media Informatics and Bachelor/Master Computer Science.