ScholarWorks@숙명여자대학교

초록

3D facial reconstruction is essential for metaverse applications. Traditional mesh-based methods have difficulty rendering photorealistic faces and complex objects. Recent advancements in Neural Radiance Fields (NeRFs) have excelled in representing complex objects. However, they struggle with capturing subtle facial differences, particularly around the eyes and mouth, due to their reliance on RGB value comparisons. To address this, we propose an EMOtion-driven Volumetric Avatar (EMOVA) that utilizes emotional stimuli from images and voices to enhance facial precision. Visual emotional features ensure that the reconstructed face aligns with the input emotion, while auditory features enhance facial details from various viewpoints. Through an attention-based fusion of these features, EMOVA accurately captures and reconstructs faces, even in self-occlusion. EMOVA outperforms the state-of-the-art methods by more than 5.93% in terms of LPIPS for face reconstruction. © 2024 Elsevier B.V.

키워드