Usability Testing for Virtual Reality: Theory & Practice

Marco Speicher, Dr. Markus Weber (Centigrade)

The seminar will be in English!

 

Time / Location

• 14:00 - 16:00, DFKI Main Building, Room Reuse

 

Important Dates

Date	Phase	Topic	Material
18.10.2018		Kick-Off	[pdf]
25.10.2018		Lecture: "Make presentations great again!" (Speicher)	[pdf1] [pdf2]
07.11.2018	I	Related Work: Paper Presentations	[zip]
15.11.2018	I	Related Work: Paper Presentations	[zip]
22.11.2018	II	Lecture: "How to conduct & report a VR study" (Speicher) Lecture: "Usability Testing" (Weber)	[pdf1] [pdf2]
06.12.2018	II	Lecture: "VR Evaluation Framework" (Speicher) Lecture: "Evaluation Metrics in Practice" (Weber)	[games] [pdf1] [pdf2]
13.12.2018	II	Study Concept: Presentations & Report Submission Deadline
10.01.2019		cancelled
17.01.2019	III	Implementation: Q&A (what do you need? how do you get it?)
24.01.2019		cancelled
31.01.2019	III	Implementation: Demo Presentations & Deadline
14.02.2019	IV	Experiments: Q&A (Open Planning Session) & Material from 22.11.
21.02.2019	IV	Lecture: "How to analyze and discuss study results" (Speicher & Weber)	[pdf1]
28.02.2019		cancelled
07.03.2019		cancelled
14.03.2019	V	Results: Presentation & Final Report Deadline (23:59)

 

Summary

Phase 1: Related Work

Each group receives various "VR User Research" papers. These should be read, summarized and presented in a 7 minute presentation per paper. Here the student should (1) present the research questions of the presentations, (2) which games/applications were used, (3) how the experiments looked like, (4) on what and how they were tested and (5) which results were achieved. The introduction is intended to give all participants an overview of how VR User Research is conducted today and what possible experiments might look like. This presentation counts for 15% of the grade.

Possible topic blocks for face-to-face events

• Usability testing in theory and practice
• Presentations of existing studies: research questions, study design, results, etc.
• Classical structure of a "study description", i.e. hypotheses, respondents, design, task, procedure, etc.
• Possible evaluation metrics (3DUI/VR or general)
• Presentation and discussion of own studies

Phase 2: Study concept

As a result, each group develops and writes a study concept that describes which research questions should be examined in its own study. This concept should include the game(s) or application(s) used, the method of the study and how the results of the study will be analysed. Each group will submit their study concept as written report and present their study concept in a 7 minute presentation, including  (1) the research questions of their study concept, (2) which games/applications could be used, (3) how the experiments will look like, (4) on what and how they will be tested and (5) which results are expected. This presentation and report count for 20% of the grade.

Potentially Interesting Aspects - Before your Own Study

• In which "error classes" can usability problems be classified? (e.g. feedback problems, presentation problems, (low level) technical problems)

Phase 3: Implementation and Study Prototype Presentation

 

Phase 4: Conducting Experiment

According to the concept, the experiment should be carried out by the group. Each seminar participant will be asked to participate in any other study (if not, you will not receive a certificate for this course), i.e. at least X participants per study.

Potentially Interesting Aspects - After the Own Study

• Which usability problems were to be expected?
• Which usability problems were surprising?
• What was simple/difficult in the organization, execution and evaluation of your own study?
• What would you do differently in the next study?

Phase 5: Analyzing & Presentation of results

After the analysis of the results, a 20-minute presentation (15 min presentation + 5 min discussion) will take place. The groups will present their project together with related papers, study concept, study description, as well as the results. This presentation serves as a final report for this seminar; if required, a written elaboration can also be submitted, which only has a positive effect on the grade, e.g. to improve the grade.

Questions about the event concept

• How are the "test objects" for the individual groups selected/assigned?
• Which infrastructure can the participants access for the tests?
• In what time period should the studies take place?
• How is this interlinked with the attendance dates? (For example: Is there a certain period of time in which the studies are carried out by the participants and in which there are weekly seminar dates in which "study-independent" things are discussed/presented?)

 

Registration

Registration by E-Mail is not possible! 

Please use the following application form for registration: https://goo.gl/forms/DTOsrkYZ0JILjxRm1 (closed)