Enhanced deep learning network for emotion recognition from GIF.

This article focuses on emotion detection in animated Graphic Interchange Format (GIF) images by combining facial expression analysis and human activity recognition using deep learning techniques. The study employs a two-stream approach integrating facial action units detected via the OpenFace library and an Inflated 3D ConvNet (I3D) with an Inception-V1 backbone for action recognition, applied to datasets including GIFGIF and Kinetics-400. Emotions are classified into seven categories: Happy, Anger, Sad, Surprise, Disgust, Fear, and Neutral, achieving an overall accuracy of 88% and an F1-score of 0.89 on a test set of 856 GIFs. Additionally, the research incorporates a Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) layers for emotion analysis of textual data, supporting a GIF recommendation system that matches text-expressed emotions with appropriate GIFs. The study highlights the challenges of emotion recognition in GIFs due to their looping, silent, and brief nature, and suggests that combining facial and action-based cues improves emotion prediction for applications in social media communication, content moderation, and sentiment analysis.