To address the satellite autonomous celestial navigation system based-on star sensor/optical camera, traditional square-root unscented Kalman filter can not well solve the nonlinear filtering problem with colored noise, which leads to the navigation system accuracy decreased. So a square-root unscented Kalman filter (CSRUKF) applied to measurement system with colored noise is proposed in this paper. In addition, in order to avoid destructing the positive and symmetry of covariance matrix caused by the errors of numerical calculation during the filtering procedure, the square-root of covariance matrix is adopted throughout recursive calculation, which improves the stability of filter. The square-root of covariance matrix update is calculated by cholesky decomposition and qr decomposition. The method was applied to satellite autonomous navigation systems. The simulation results show that, compared to traditional SRUKF, this proposed SRUKF can well solve the problem of poor estimation accuracy in measurement system with colored noise.