ScholarWorks@숙명여자대학교

초록

Multimodal Emotion Recognition (MER) achieves precise emotion prediction by leveraging diverse modalities suchas audio, visual, and textual cues. However, in real-world scenarios, missing modalities frequently occur due tosensor malfunctions or privacy concerns, leading to significant degradation in model performance. Although variousapproaches have attempted to address this issue by generating missing data or reconstructing latent features fromavailable modalities, most rely on simple statistical mappings or numerical approximations. Consequently, theyoften fail to capture linguistic contexts or complex semantic interactions between modalities. In this paper, wepropose a feature reconstruction framework that leverages the powerful reasoning capabilities of Large LanguageModels (LLM). The proposed model utilizes an LLM as a feature space encoder to semantically complement missingmodalities. Furthermore, it employs a parallel cross-attention mechanism to effectively fuse information acrossdifferent modalities. Extensive experiments demonstrate the validity and effectiveness of our proposed methodunder incomplete data conditions.

키워드