Usability Testing for Virtual Reality: Theory & Practice
Marco Speicher, Dr. Markus Weber (Centigrade)
The seminar will be in English!
Time / Location
- 14:00 - 16:00, DFKI Main Building, Room Reuse
|25.10.2018||Lecture: "Make presentations great again!" (Speicher)||[pdf1] [pdf2]|
|07.11.2018||I||Related Work: Paper Presentations||[zip]|
|15.11.2018||I||Related Work: Paper Presentations||[zip]|
|22.11.2018||II||Lecture: "How to conduct & report a VR study" (Speicher)
Lecture: "Usability Testing" (Weber)
|06.12.2018||II||Lecture: "VR Evaluation Framework" (Speicher)
Lecture: "Evaluation Metrics in Practice" (Weber)
|[games] [pdf1] [pdf2]|
|13.12.2018||II||Study Concept: Presentations & Report Submission Deadline|
|17.01.2019||III||Implementation: Q&A (what do you need? how do you get it?)|
|31.01.2019||III||Implementation: Demo Presentations & Deadline|
|14.02.2019||IV||Experiments: Q&A (Open Planning Session) & Material from 22.11.|
|21.02.2019||IV||Lecture: "How to analyze and discuss study results" (Speicher & Weber)||[pdf1]|
|14.03.2019||V||Results: Presentation & Final Report Deadline (23:59)|
Phase 1: Related Work
Each group receives various "VR User Research" papers. These should be read, summarized and presented in a 7 minute presentation per paper. Here the student should (1) present the research questions of the presentations, (2) which games/applications were used, (3) how the experiments looked like, (4) on what and how they were tested and (5) which results were achieved. The introduction is intended to give all participants an overview of how VR User Research is conducted today and what possible experiments might look like. This presentation counts for 15% of the grade.
Possible topic blocks for face-to-face events
- Usability testing in theory and practice
- Presentations of existing studies: research questions, study design, results, etc.
- Classical structure of a "study description", i.e. hypotheses, respondents, design, task, procedure, etc.
- Possible evaluation metrics (3DUI/VR or general)
- Presentation and discussion of own studies
Phase 2: Study concept
As a result, each group develops and writes a study concept that describes which research questions should be examined in its own study. This concept should include the game(s) or application(s) used, the method of the study and how the results of the study will be analysed. Each group will submit their study concept as written report and present their study concept in a 7 minute presentation, including (1) the research questions of their study concept, (2) which games/applications could be used, (3) how the experiments will look like, (4) on what and how they will be tested and (5) which results are expected. This presentation and report count for 20% of the grade.
Potentially Interesting Aspects - Before your Own Study
- In which "error classes" can usability problems be classified? (e.g. feedback problems, presentation problems, (low level) technical problems)
Phase 3: Implementation and Study Prototype Presentation
Phase 4: Conducting Experiment
According to the concept, the experiment should be carried out by the group. Each seminar participant will be asked to participate in any other study (if not, you will not receive a certificate for this course), i.e. at least X participants per study.
Potentially Interesting Aspects - After the Own Study
- Which usability problems were to be expected?
- Which usability problems were surprising?
- What was simple/difficult in the organization, execution and evaluation of your own study?
- What would you do differently in the next study?
Phase 5: Analyzing & Presentation of results
After the analysis of the results, a 20-minute presentation (15 min presentation + 5 min discussion) will take place. The groups will present their project together with related papers, study concept, study description, as well as the results. This presentation serves as a final report for this seminar; if required, a written elaboration can also be submitted, which only has a positive effect on the grade, e.g. to improve the grade.
Questions about the event concept
- How are the "test objects" for the individual groups selected/assigned?
- Which infrastructure can the participants access for the tests?
- In what time period should the studies take place?
- How is this interlinked with the attendance dates? (For example: Is there a certain period of time in which the studies are carried out by the participants and in which there are weekly seminar dates in which "study-independent" things are discussed/presented?)
Registration by E-Mail is not possible!
Please use the following application form for registration: https://goo.gl/forms/DTOsrkYZ0JILjxRm1 (closed)