5BHITN 2021/22
„MARC“ is a barrier-free digital assistant with a focus on privacy and data security. It controls various devices and parameters in the working environment. Voice and gesture inputs support people with disabilities. A digital avatar makes MARC more lifelike.
Idea:
Poor ergonomics, a lack of assistance and little social contacts limit the work capacity at home offices. Therefore, people need a secure, accessible, and practical solution to take load off them.
Solution:
Gesture control was implemented using the Grove Gesture PAJ7620 sensor which can recognize 9 different hand gestures. Speech recognition is based on the open-source software „Vosk“ because of its offline functionality and for the extensive language training models. To realize the profile management, the silent alarm and the focus tracker, face recognition was added. This was implemented using the software „Face-API.js“ and self-made training models for the faces to be recognized. The avatar is a model created in Blender, which adds a robotic face to the assistant „MARC“, but at the same time contains the human features. The web interface is intended as a mobile interface between the user and „MARC“. This runs on an Apache server and communicates with the assistant via a database.
Demonstration video: click here
Team: Whdan AHMED, Abdul FAQIRZADA, Almin HUREMOVIC, Levent GÜNEL, Philipp RAUTNER, Umut ILHAN
Supervisor: Mag. Harald Dassler
