A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations

Soumya Dutta, Smruthi Balaji, Sriram Ganapathy|February 26, 2026arXiv

Key Takeaway

Using specialized experts for different modalities (speech vs.

Summary

This paper presents MiSTER-E, a system that recognizes emotions in conversations by combining speech and text information. It uses separate AI experts for speech, text, and cross-modal analysis, then intelligently combines their predictions. The system works on real conversations without needing to know who's speaking, and achieves strong results on standard emotion recognition benchmarks.

multimodal architecture applications

Key Terms

mixture-of-experts multimodal-fusion fine-tuning contrastive-loss