A Theory-Based Explainable Deep Learning Architecture for Music Emotion.

This article presents the development of a theory-based, explainable deep learning convolutional neural network (CNN) classifier designed to predict the time-varying emotional response to music using the valence-arousal framework, where valence indicates positivity and arousal indicates energy level. The model incorporates novel CNN filters structured around the harmonic frequencies fundamental to Western tonal music, enabling it to capture midlevel musical features such as consonance, which is closely linked to emotional perception. Compared to atheoretical deep learning models and those using handcrafted features, the harmonics-based model achieves comparable predictive accuracy with greater parsimony and enhanced explainability, demonstrated through gradient-weighted class activation mapping (Grad-CAM) visualizations. The practical utility of the model is illustrated in a laboratory experiment simulating YouTube midroll ad insertions, showing that ads placed in emotionally congruent contexts—identified via the model’s predictions—yield higher engagement (lower skip rates and higher brand recall) than emotionally incongruent placements. This research contributes to marketing and machine learning literature by providing an interpretable tool for emotion-based music analysis with applications in digital advertising and content recommendation.