Water pollution seriously affects the water landscape and water ecology. In this study, a deep-wise convolution and cross attention (DCCA) algorithm module is proposed to address the issues of complex water surface scenes and difficulty in extracting features of small target pollutants in the process of identifying water surface pollution. The use of deep-wise convolution reduces the parameters and computational complexity of the model, and establishes relationships between feature maps at different scales using cross attention, enabling the model to better understand contextual information and improve its ability to recognize complex scenes and small targets. The experimental results show that the average accuracy has been improved by 1.8% after adding the DCCA module, reaching 88.7%. The detection effect of water surface pollution has been improved by using less memory occupation.