A Hybrid Architecture for Music Recommendation and Control: Integrating Facial Emotion Recognition and Hand Gesture Interaction

Key Points

• The aim is to develop a music recommendation system that engages users in real time using facial emotion recognition and hand gestures.
• Designed a music recommendation system integrating facial emotion recognition and hand gesture control.
• Employed convolutional neural networks for emotion classification based on facial expressions.
• Utilized Media Pipe for hand gesture recognition using 21 landmarks.
• Implemented preprocessing techniques for reliable performance under various conditions.
• Evaluated performance using metrics like recommendation relevance and gesture recognition accuracy.
• Achieved improved recommendation relevance through real-time emotion analysis.
• Demonstrated high accuracy in gesture recognition for play, pause, skip, and volume control.
• Ensured low end-to-end latency, enhancing user experience.

Abstract