Motivation
Die Eyeled GmbH hat sich als Anbieter von Apps für Weltleitmessen etabliert. Die Apps enthalten alle relevanten Informationen für den Besucher zur Vorbereitung und für den Messebesuch selbst.
Die jüngsten Fortschritte bei den großen Sprachmodellen (Large Language Models, LLMs) ermöglichen es inzwischen, natürlichsprachliche Fragen basierend auf der vorhandenen Datenlage kompetent zu beantworten.
In einer vorangegangenen Master-Arbeit wurde bereits ein System entwickelt, welches basierend auf ausgewählten Ausstellern und Terminen auf dem Messegelände einen optimalen Weg berechnen kann.

Thema
Basierend auf einem LLM und den bereits geleisteten Vorarbeiten soll ein persönlicher Assistent entwickelt werden, der einerseits konkrete Fragen rund um den Messebesuch beantworten kann, andererseits aber auch basierend auf den Interessen des Nutzers einen Vorschlag für einen Besuchsplan erstellen kann und diesen durch Feedback des Nutzers iterativ optimieren kann.

Zu untersuchen ist, inwieweit die Antworten des LLMs automatisch ausgewertet werden können, sodass z. B. vorgeschlagene Aussteller automatisch erkannt werden und in die Wegeplanung aufgenommen werden können. Dadurch kann basierend auf der Wegberechnung auch automatisch Feedback an das LLM gegeben werden, wenn z. B. die vorgeschlagenen Aussteller innerhalb einer vorgegebenen Zeit nicht besucht werden können, weil sie zu weit über das Messegelände verteilt liegen.

Inhalte der Arbeit

• Evaluation des derzeitigen Stands der Forschung
• Analyse der Anwendungsfälle
• Konzeption des Systems
• Prototypische Umsetzung und Evaluierung
• Schnittstellendefinition und Integration mit bestehender Messe-App (Anpassungen an Messe-App ist nicht Teil der Anforderungen) 

Voraussetzungen

• Interesse am Thema Künstlicher Intelligenz und insbesondere LLMs
• Gute bis sehr gute Programmierkenntnisse
• Datenbankkenntnisse (SQLite)

Bewerbung
Bitte sende eine E-Mail an Frederic Kerber mit den folgenden Informationen:

• Gewünschter Start der Arbeit
• Geplante Fertigstellung der Arbeit
• Ein kurzes Motivationsstatement, warum das Thema interessant für dich ist
• Eine kurze Zusammenfassung, warum du gut für die Bearbeitung des Themas geeignet bist
• Übersicht besuchter Lehrveranstaltungen

Ansprechpartner

DFKI: Frederic Kerber
Eyeled GmbH: Christoph Bartelmus

In many “in the wild” studies you are interested in the thoughts and feelings of participants at specific points in time (e.g., multiple times a day). Often, such studies use a (in-situ) diary approach. In this work you should explore means to motivate people to engage in this often tedious process. Although your approach should work independent of the actual data that participants have to provide, one context that we are especially interested in is the health domain: here, participants should provide data on how they feel (multiple times per day). This work is suitable for a Bachelor’s or Master’s thesis.

Focus

You have to implement a prototype using web technology, and you should conduct (at least) one study to verify whether your approach works.

Prerequisites

• Read papers in respect to such diary approaches. In the recent years, some suggestions were already made at HCI-conferences (hint: start with the proceedings of the last CHI conferences).
• Experience with web development.
• Completed HCI and/or Statistics lecture and ideally already attended at least one seminar at our chair.

How to apply

Please send me an email with the following pieces of information (if you do not answer every point, your application will not be considered):

• When you plan to start the thesis
• When you plan to finish the thesis
• A short motivational statement why this topic is interesting for you
• A summary why you be a good fit for this topic
• Your transcript of records
• A list of relevant papers that you have read and what you learned from them. Additionally, indicate how you conducted the search (max 2 A4 pages)
• A brief idea description: which motivational approach would you follow and why do you think that this is reasonable?

This work focuses on human-robot collaboration, in more detail, how a robotic arm and a human can work together at an assembly cell such that the robot pro-actively supports the worker in assembling a workpiece. A prototypical set-up including the robot and components of the workpiece are already available as well as a mixed-reality duplicate of the set-up, which can be used to conduct Wizard-of-Oz (WoZ) style user-studies using AR glasses. Building on the existing work, your task is to conceptualize, plan, conduct and assess a user-study to determine the most appropriate work dynamic (the optimal division of tasks) between the robot and the worker. This also includes determining the most suitable modalities for human-robot communication during the process. This thesis is a collaboration between ZeMA (Zentrum für Mechatronik und Automatisierungstechnik gGmbH) and DFKI. The practical work will be done at the Power4Production Hall at Eschbergerweg 46, Saarbrücken.

Focus

The focus is on the user-study itself. You will need to create a storyline, think about relevant questions, including identifying which data must be recorded, find participants, plan the execution, and analyze the results. 

Prerequisites

• Background in planning and conduction user-studies (e.g. from the HCI lecture)
• Interest in Mixed Reality (e.g. Meta Quest AR headsets)
• Interest in Robotics
• Enrolled in a Bachelor programme in computer science, mediainformatics or related field

How to apply

Please send us an email with the following pieces of information (if you do not answer every point, your application will not be considered):

• When you plan to start the thesis
• When you plan to finish the thesis
• A short motivational statement why this topic is interesting for you
• A summary why you would be a good fit for this topic
• Your transcript of records and CV

Task-oriented grasping is a well-known problem in the robotics domain. Modeling the complex relation between grasping types, objects, and the intended task is a long-studied paradigm. However, most existing methods still focus on closed-world concepts and non-anthropomorphic solutions using two-finger grippers. More recently, researchers incorporated Large Language Models (LLMs) to analyze and describe objects based on tasks for a grasping robot. In this thesis, you will focus on an LLM-based solution for a five-finger anthropomorphic robotic hand that goes beyond the limitation of existing semantic knowledge-based methods.
Moreover, you would combine this information with vision-based input/geometric data, predict a grasp type, and plan/execute the grasp. You should do this in a simulation since this is relatively complicated and focus only on the most fundamental grasp types. You can refer to the following two papers (among others you might find yourself) that focus on this problem.

Prerequisites

• Please read about the following papers [1] [2] [3]
  
• Familiar with deep learning concepts, Large Language Models, and Simulation environments (e.g., Unity)
• Completed Statistics, Machine learning, and/or HCI courses
• Strong programming skills

How to apply

Please send us an email with the following pieces of information:

• When you plan to start the thesis
• When you plan to finish the thesis
• A short motivational statement why this topic is interesting for you
• A summary why you would be a good fit for this topic
• Your transcript of records and CV

Recent advances in Machine learning (specifically Deep Learning) allowed robots to understand objects and the surrounding environment on a perceptual non-symbolic level (e.g. object detection, sensors fusion, and language understanding), however a trending area of research is to understand objects on a conceptual symbolic level so we can achieve a level of robots thinking like humans. Deep Reinforcement Learning (RL) recently attempted implicitly combining these symbolic and non-symbolic learning paradigms, but it has several drawbacks such as: (1) the need for very long training time with respect to traditional deep learning approaches, (2) convergence to optimum policy is not guaranteed and it can get stuck in a sub-optimal policy, and (3) a RL agent is trained over a simulated environment so it cannot foresee actions that only exist in the real environment. The goal of thesis is to train a robot that would explicitly learn on both perceptual and conceptual levels through direct feedback from a human expert along with its existing view (i.e. sensors) of the world.

Focus

This work will focus on Reinforcement Learning, Imitation Learning and the combination of both. This work will involve real-time implementation of a working system.

Prerequisites

• Please read about the following papers [1] [2] [3] [4] [5]
  
• Background or interest in RL, Computer Vision or AI Planning
• Completed HCI, Statistics and/or Machine learning courses
• Strong programming skills
• Unity/SImulation environments background is a plus

How to apply

Please  send me an email  with the following pieces of information:

• When you plan to start the thesis
• When you plan to finish the thesis
• A short motivational statement why this topic is interesting for you
• A summary why you would be a good fit for this topic
• Your transcript of records and CV

Referencing resolution is a trending topic that remains unsolved due to the high variance in users' behavior when performing a referencing task. Referencing resolution is simply identifying the object a user is intending to select through speech, pointing, gaze or multi modal fusion of all the previous modalities. Referencing is used in multiple domains in HCI such as Human Robot Interaction (HRI) [Nickel et al. 2003; Whitney et al. 2016; Kontogiorgos et al. 2018; Sibirtseva et al. 2019], and Vehicle and Drone interaction [Rümelin et al. 2013; Roider et al. 2017; Gomaa et al. 2020]. However, most of the current research focus on stationary first-person view when interacting with the object. In this thesis, you will work on the task of multi-modal real-time reference resolution using speech, gaze and/or pointing gestures from a moving source when interacting with a vehicle, industrial robot, or retail delivery drone. 

Focus

This work will focus on gesture identification, gaze tracking, object detection, speech recognition and/or modality fusion techniques. This work will involve real-time implementation of a working system.

Prerequisites

• Please read about the following papers [1] [2] [3] [4] [5] [6]
• Background or interest in gesture recognition, NLP or gaze tracking
• Completed HCI, Statistics and/or Machine learning courses
• Strong programming skills

How to apply

Please  send me an email  with the following pieces of information:

• When you plan to start the thesis
• When you plan to finish the thesis
• A short motivational statement why this topic is interesting for you
• A summary why you would be a good fit for this topic
• Your transcript of records and CV