ADVANCING HUMAN-ROBOT INTERACTION: DEEP LEARNING-BASED EMOTION AND GESTURE RECOGNITION FOR IVASTBOT

Abstract

In this paper, we introduce a novel approach to enhance the capabilities of the humanoid robot IVastBot by in- tegrating various software components. This integration enables IVastBot to effectively recognize and respond to a wide array of human gestures and behaviors. Through the utilization of the open-source MediaPipe Pose library and LSTM networks, IVastBot becomes proficient in generating contextually appropri- ate responses. Furthermore, we incorporate emotion recognition into the system using Convolutional Neural Networks (CNN). The entire recognition module seamlessly integrates into the Robot Operating System (ROS) architecture, resulting in efficient execution. Consequently, IVastBot achieves the ability to execute adaptive actions in response to human gestures and emotions, sig- nificantly enriching the intuitiveness and engagement of human- robot interactions