Low-rank sparse coding and region of interest pooling for dynamic 3D facial expression recognition

Abstract

In this paper, we propose a dynamic three-dimensional facial expression recognition using low-rank sparse codes pooled from automatically detected regions of interests. Low-rank sparse coding has been applied for the first time in dynamic facial expression recognition and a novel region-based pooling is suggested. Twelve regions of interests are defined using detected landmarks based on facial expression activation dynamics. Landmarks are tracked in subsequent frames using multi-point tracker. Temporal segmentation is utilized by using geometric information extracted from detected landmarks. LBP-TOP feature descriptors are extracted from cuboids inside spatiotemporal regions of interests in both texture and depth sequences. Texture and depth features are then fused to form the feature matrix. Low-rank sparse coding is utilized to obtain sparse codes from feature descriptors. Finally, hidden-state conditional random fields are employed to classify six basic expressions. Experimental results and comparison with recent studies verify that proposed method improves the accuracy of dynamic facial expression recognition. The average accuracy of recognition of six basic expressions is improved by the proposed system from 83.07 to 85.12% on 95 subjects in BU-4DFE data set.