In view of complex networks, large computation amount, and high hardware platform requirements in the current process of applying deep learning to realize digital signal modulation and recognition, this study proposes a method of using signal constellation diagram modulation and recognition in the improved MobileNetV3 lightweight neural network. Firstly, the received MPSK and MQAM signals are converted into constellation diagrams, which are extracted from gray images, and gray images are enhanced. The image dataset of constellation diagrams is then constructed, and the cross-layer structure of ResNet is introduced into the MobileNetV3 network. As a result, the phenomenon of vanishing gradient caused by decreasing weight with increasing network layers is solved. Finally, the dataset of constellation diagrams is used to train the weight of the MobileNetV3 lightweight neural network, and then the constellation diagrams are recognized. MobileNetV3 greatly reduces the number of parameters and training time on the premise of ensuring recognition accuracy based on deep convolution separable technology and network architecture search (NAS) technology. For the modulation and recognition of simple signals, the lightweight neural network can effectively simplify the network structure and reduce hardware requirements. The simulation results show that the modulated signals (BPSK, QPSK, 8PSK, 16QAM, and 64QAM) can achieve modulation and recognition with a recognition rate of 99.76%. Compared with traditional networks using deep learning to realize modulation and recognition, the lightweight neural network can significantly reduce the number of network parameters and computational costs.