The images combining visible and infrared light would produce color deviation. In response to this phenomenon, this study proposes a de-crosstalk matrix color correction algorithm based on polynomial regression to counter the crosstalk problem of four-band images (RGB three-band and IR near-infrared band). Under specific lighting conditions, the algorithm uses 24 color blocks of standard color checker to build the de-crosstalk matrix according to the idea of polynomial regression. The de-crosstalk matrix will be used for four-band images which are taken under the same lighting conditions to remove the near-infrared crosstalk and realize color correction. The experimental results indicate that the method does not only need to calibrate the correction matrix at one time, the color deviation images taken under the same lighting conditions will be corrected and restored the natural colors of themselves thereafter.