Aiming at the complexity of background image in object detection and the singleness of object’s color type in common back-projection method, an optimized object detection method based on histogram back-projection was presented. In this algorithm, multiple object models were established to make up the defect that multiple objects with different colors cannot be detected by using common back-projection method. Besides, in order to reduce the interference of complex background, a series of optimizing process steps for the probability image was added. Experimental results show that the proposed algorithm not only can accurately detect the position of objects in real-time, but also with better detection effect and higher operation efficiency than the common back-projection algorithm and other methods.