Amr Gomaa

Doctoral Researcher

Loading...


Research Interests

Computer VisionApplied Machine LearningDeep LearningReinforcement LearningMulti-modal InteractionHuman Computer InteractionAdaptive Interfaces
 

Open Thesis Topics

Recent advances in Machine learning (specifically Deep Learning) allowed robots to understand objects and the surrounding environment on a perceptual non-symbolic level (e.g. object detection, sensors fusion, and language understanding), however a trending area of research is to understand objects on a conceptual symbolic level so we can achieve a level of robots thinking like humans. Deep Reinforcement Learning (RL) recently attempted implicitly combining these symbolic and non-symbolic learning paradigms, but it has several drawbacks such as: (1) the need for very long training time with respect to traditional deep learning approaches, (2) convergence to optimum policy is not guaranteed and it can get stuck in a sub-optimal policy, and (3) a RL agent is trained over a simulated environment so it cannot foresee actions that only exist in the real environment. The goal of thesis is to train a robot that would explicitly learn on both perceptual and conceptual levels through direct feedback from a human expert along with its existing view (i.e. sensors) of the world.

 

Focus

This work will focus on Reinforcement Learning, Imitation Learning and the combination of both. This work will involve real-time implementation of a working system.

 

Prerequisites

  • Please read about the following papers [1] [2] [3] [4] [5]
  • Background or interest in RL, Computer Vision or AI Planning
  • Completed HCI, Statistics and/or Machine learning courses
  • Strong programming skills
  • Unity/SImulation environments background is a plus

 

How to apply

Please  send me an email  with the following pieces of information:

  • When you plan to start the thesis
  • When you plan to finish the thesis
  • A short motivational statement why this topic is interesting for you
  • A summary why you would be a good fit for this topic
  • Your transcript of records and CV

Referencing resolution is a trending topic that remains unsolved due to the high variance in users' behavior when performing a referencing task. Referencing resolution is simply identifying the object a user is intending to select through speech, pointing, gaze or multi modal fusion of all the previous modalities. Referencing is used in multiple domains in HCI such as Human Robot Interaction (HRI) [Nickel et al. 2003; Whitney et al. 2016; Kontogiorgos et al. 2018; Sibirtseva et al. 2019], and Vehicle and Drone interaction [Rümelin et al. 2013; Roider et al. 2017; Gomaa et al. 2020]. However, most of the current research focus on stationary first-person view when interacting with the object. In this thesis, you will work on the task of multi-modal real-time reference resolution using speech, gaze and/or pointing gestures from a moving source when interacting with a vehicle, industrial robot, or retail delivery drone. 

 

Focus

This work will focus on gesture identification, gaze tracking, object detection, speech recognition and/or modality fusion techniques. This work will involve real-time implementation of a working system.

 

Prerequisites

  • Please read about the following papers [1] [2] [3] [4] [5] [6]
  • Background or interest in gesture recognition, NLP or gaze tracking
  • Completed HCI, Statistics and/or Machine learning courses
  • Strong programming skills

 

How to apply

Please  send me an email  with the following pieces of information:

  • When you plan to start the thesis
  • When you plan to finish the thesis
  • A short motivational statement why this topic is interesting for you
  • A summary why you would be a good fit for this topic
  • Your transcript of records and CV

Vehicles get exponentially smarter every day; Manufacturers are continuously adding more features to smart cars. While these functionalities are added to enhance driver's experience and make their rides smoother, they often come with extra complexity that causes more stress and might make the trip more dangerous. Several researchers attempted conceptualizing / introducing situation-aware personalized adaptive interfaces that would ultimately reduce the interface complexity [Garzon et al. 2010; Garzon et al. 2011; Garzon 2012; Siegmund et al. 2013; Walter et al. 2015; Hasenjäger et al. 2017; Knauss et al. 2018]. However, as far as we know, there is no actual implementation for such interface due to the lack and obtaining difficulty of such training data. 

 

Focus

In this thesis, you will work on a two-stage project that 1) Identify certain activities or scenarios based on driving behavior for the use in traditional or autonomous driving situations using state diagrams or specific schema, 2) Use a hybrid deep learning approach (e.g., Graph Neural Networks or Deep Reinforcement Learning) to find adaptive patterns in behavior using small amount of data. You will focus on activity recognition, situation awareness and hybrid learning approaches combining symbolic and sub-symbolic learning.

 

Prerequisites

  • Please read about the following papers [1] [2] [3]  [4] [5] [6]
  • Familiar with deep learning concepts and / or state diagrams / graphical modeling
  • Completed AI Planning, Statistics and / or Machine learning courses
  • Strong programming skills

 

How to apply

Please   send me an email   with the following pieces of information:

  • When you plan to start the thesis
  • When you plan to finish the thesis
  • A short motivational statement why this topic is interesting for you
  • A summary why you would be a good fit for this topic
  • Your transcript of records and CV
 

HiWi Position in Automotive Embedded Systems area

With the exponential advancements of modern vehicles, modern cars (specifically smart cars) possess many sensors and collect much private information about the drivers. Insurance companies collect this data through special dongles and utilize this information to judge your driving skills and driving style to determine which payment plan they should assign to you. The nature of this data and the way it is stored is still a mystery to the public. In this work, you will reverse engineer such mechanisms for collecting the data employed by these insurance companies and study the privacy issues it entails. Additionally, you would investigate the current knowledge of a sample population through proper study design and statistical analysis. This work will involve related work surveying, system implementation, study design and analysis, and technical writing.

Prerequisites:

  • Please read the following papers [1], [2], [3], and [4] and be prepared to present them
  • Background in Embedded Systems, Automotive, and Reverse Engineering
  • Completed Embedded Systems course
  • Strong programming skills
  • Completed Software Engineering, Embedded Security, HCI, or Statistical Analysis courses are a plus

How to apply:

Please send us (amr.gomaa@dfki.de and singh@cispa.de) an email with the following pieces of information:

  • A short motivational statement about why this topic is interesting for you
  • A summary of why you would be a good fit for this topic
  • Your transcript of records and CV

 

Projects

CAMELOT

TeachTAM

TRACTAT

APX-HMI

Teaching

Seminar: Speech-based Adaptation of Personalized User Interfaces (Winter 2022/2023)

Adaptive Human Machine Interfaces for Autonomous Systems (Winter 2021/2022)

Hybrid Machine Learning Approaches and Applications (Winter 2020/2021)

Seminar: Automotive User Interfaces (Winter 2019/2020)

Academic Services

Reviewer: CHI 2022, IEEE VR 2022, NordCHI 2022, AutomotiveUI 2022 and 2021, CHI PLAY 2021, ICMI 2021, and IEEE AIVR 2021


Publications ()

Loading...