Deep Joint Spatiotemporal Network (DJSTN) for Efficient Facial Expression Recognition
- Authors
- Jeong, Dami; Kim, Byung-Gyu; Dong, Suh-Yeon
- Issue Date
- Mar-2020
- Publisher
- MDPI
- Keywords
- facial expression recognition (FER); deep learning; local binary pattern (LBP) feature; geometric feature; deep spatiotemporal network; joint fusion classifier
- Citation
- SENSORS, v.20, no.7
- Journal Title
- SENSORS
- Volume
- 20
- Number
- 7
- URI
- https://scholarworks.sookmyung.ac.kr/handle/2020.sw.sookmyung/1525
- DOI
- 10.3390/s20071936
- ISSN
- 1424-8220
1424-3210
- Abstract
- Understanding a person's feelings is a very important process for the affective computing. People express their emotions in various ways. Among them, facial expression is the most effective way to present human emotional status. We propose efficient deep joint spatiotemporal features for facial expression recognition based on the deep appearance and geometric neural networks. We apply three-dimensional (3D) convolution to extract spatial and temporal features at the same time. For the geometric network, 23 dominant facial landmarks are selected to express the movement of facial muscle through the analysis of energy distribution of whole facial landmarks. We combine these features by the designed joint fusion classifier to complement each other. From the experimental results, we verify the recognition accuracy of 99.21%, 87.88%, and 91.83% for CK+, MMI, and FERA datasets, respectively. Through the comparative analysis, we show that the proposed scheme is able to improve the recognition accuracy by 4% at least.
- Files in This Item
-
Go to Link
- Appears in
Collections - ICT융합공학부 > IT공학전공 > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.