Individual electroencephalogram (EEG) signals are different and vulnerable to environmental factors. In view of these problems, this study adopts methods of removing baseline interference and EEG channel selection and proposes an emotion classification and recognition algorithm based on a continuous convolutional neural network (CNN). Firstly, the selection of differential entropy (DE) characteristics of baseline signals is studied. After the data is processed into multi-channel input, the continuous CNN is used for classification experiments, and then the optimal number of electrodes is determined. The experimental results show that after the difference between the DE of experimental EEG signals and that of baseline signals of the subject one second before the experimental EEG is mapped into a two-dimensional matrix, and the frequency dimension is turned into a multi-channel form to serve as the input of the continuous CNN, the average classification accuracy of arousal and valence on 22 channel is 95.63% and 95.13%, respectively, which are close to that on 32 channel.