Cloud occlusion in optical remote sensing images is one of the core challenges in remote sensing data processing. To address the limitations of current cloud removal technologies in handling cloud edge information and preserving image details, a generative adversarial network (TGAN) based on temporal-spectral domain fusion and temporal self-attention enhancement is proposed. Through its two-stage modular design, TGAN simultaneously improves the quality of remote sensing image restoration and processing efficiency. In the first stage, the feature extraction module, based on a temporal self-attention mechanism, uses a linear expansion layer to capture temporal and spectral domain features, compensating for the limitations of traditional maximum pooling with a one-dimensional linear dimensionality reduction layer, thus enhancing the modeling capability of time-series positional features. A multi-head self-attention mechanism with a weight allocation strategy is designed to accurately capture edge information. The second stage is an adaptive image restoration module, composed of a random noise cancellation submodule and a local contrast enhancement submodule, which collaboratively enhances image details and suppresses noise. In addition, TGAN’s discriminator incorporates multi-scale modules, a strategy that balances global consistency and local detail. Through the interactive game between the generator and discriminator, the generator continuously optimizes the restored image, improving restoration performance. This dynamic adversarial process drives iterative optimization of the generator in the image restoration task. To verify the effectiveness of TGAN, experiments are conducted on the Sen2_MTC dataset. The results show that TGAN significantly outperforms existing methods in terms of peak signal-to-noise ratio (PSNR) and subjective visual evaluation, with PSNR values of 21.547 dB and 20.206 dB for the training and test sets, respectively, indicating that TGAN demonstrates strong performance and application potential in remote sensing cloud image restoration.