The fiber optic gyroscope is an inertial instrument, which is used widely, but there are still many problems in the application of the FOG. The disadvantages of the fiber optic gyroscopes such as such as temperature effect, non-linearity of scale factory and disturbance. This paper analyses the principle and the signal output of the fiber-optic gyroscope. It establishes the compensate model of nonlinearity error and linearity error, uses signal processing algorithm and signal filtering algorithm to compensate variety of error. The signal processing algorithm is proved to be effective through actual experiments.