In the process of video or image transmission, there may be random error, sudden error, packet loss, and so on, which will also have a serious impact on the decoded image data. This paper presents an image reconstruction algorithm based on depth learning:an unsupervised image reconstruction neural network model based on image background prediction to generate fuzzy region content. In order to reconstruct a vivid image, a neural network model not only needs to understand the content of the image, but also to reconstruct the missing part of a reasonable assumption. The loss function includes standard pixel level reconstruction loss and counterwork loss. When training the convolution neural network model, the loss function can better deal with the structure details in the image and produce clearer results. Through experiments, we can find that the neural network model of depth convolution designed in this study has better effect in image reconstruction than the algorithm based on sample interpolation.