Abstract:Traditional facial expression recognition (FER) methods have focused on the six basic facial expressions. However, compound facial expressions are also used by humans in the real world. Compound facial expression means that it is a combination of the basic facial expressions. However, the traditional methods of recognizing basic facial expressions are unable to handle compound facial expressions. Moreover, the compound facial expression datasets have insufficient training data. To address the difficulties in the compound FER, this study proposed a graph convolutional network in multi-label learning for compound facial expression recognition (GCN-ML-CFER). The global features of facial expression and the local features of the regions of interest were extracted by the feature extraction network. According to the prior knowledge of basic and compound facial expressions, a relationship graph of facial expression categories was constructed by a data-driven method. The expression category classifiers learn the graph via a graph convolutional network (GCN). Finally, compound FER was carried out by the classifiers. Experiments were conducted on the RAF-DB and EmotioNet datasets. The results show that this method achieves a 4%–5% increase in the compound FER accuracy compared with those of the VGG19 and ResNet50 methods.