The method of using software to decoding image cannot satisfy the timely request. This paper proposes a system design plan of JPEG image decompression. It mainly gives the JPEG image decompression algorithm’s hardware realization and system’s principle of work as well as the hardware realization. The result indicates that this system can speed up the decoding speed of image only with 21.3ms when decoding a 640×320 image. It realizes the image decoding timely.