Machine Learning, and Deep Learning, in particular, have improved the recognition of dynamic hand gestures in the past decade. Different architectures like CNNLSTMs [1], 3D Convolutional Networks [2], and others have been developed to approach this task. However, one of the problems that these architectures have, is that they require large amounts of data in order to produce high-performance models. This data might not be available in particular situations like different camera angles (e.g., top-down), or different types of cameras (e.g., time-of-flight cameras). While transfer learning can sometimes be used to reduce the amount of data required to train these models, the available pre-trained models may sometimes be unsuitable and not help solve the new task. An alternative to this is the use of synthetic data. By providing synthetic data that is similar to the one that is expected in the real world, the amount of new data that needs to be collected decreases. For this purpose, modern game engines and modeling software could be used to predefine the gestures and introduce small variations [3]. The aforementioned architectures could then use this data in combination with real-world gestures to train the gesture recognition model from scratch.  

Focus

The student will develop a dynamic gesture generation system using modern game engines like Unity3D or Unreal Engine. The evaluation of this system will depend on the type of thesis the student pursues (Bachelor’s or Master’s thesis). 

Prerequisites