This paper analyzes the algorithm of Inverse Transform and Quantization(ITIQ) in H.264 and proposes a high performance architecture. A reconfigurable one dimensional IDCT (Inverse Discrete Cosine Transform) architecture of Inverse Transform is proposed; A 4-IQ-parallel architecture of Inverse Quantization is adopted, area-consuming MODULO and DIVISION operation of Inverse Quantization are implemented by LUTs instead of direct computation. Unified clock gating scheme and pipeline technology are also used in the design. The results show that the design increases the system throughput and drops the power consumption and meets the real-time decoding of 1080i stream.