For better performance of the projection mapping technology applied to the digital exhibition of museums and high-quality projection, a 3D digital display system for cultural relics is designed with realistic 3D modeling and a high-fidelity appearance. The system relies on the physical model of mechanical rotation and the image of real cultural relics and combines multiple projection mapping technologies with the optical see-through display. Both geometric and radiometric calibration methods are used to correctly project a high-quality texture onto the moving 3D projection surface without perspective distortion to increase the projection effect and the degree of freedom of the projection content. For the evaluation of the influence of the ambient light and the shadow occlusion by the surrounding objects in general museums on the system, the projection area and display area of the system are tested with different intensities of light. Experimental results show that the system has strong robustness to the influence of the ambient light and the potential for deployment in a general museum context, making the digital display of cultural relics richer.