Abandoned objects detection is the core function of the intelligent video surveillance system. Abandoned objects are generally small and in the complex environment, the traditional moving target detection algorithms usually cannot well detect abandoned objects directly. This paper proposes an abandoned objects detection algorithm based on frame difference and edge difference. Firstly, the algorithm gets the moving target area by frame difference calculation. Then, it gets the edge of the moving target by edge difference calculation between the current frame image and the previous frame background image. It merges this two difference calculation result to get the complete moving target edge features. Finally, if the residence time of this moving target in the scene meets or exceeds the threshold set by the alarm system, this paper marks this target as an abandoned object and the intelligent video surveillance system will process this result. The experiment results show that the proposed algorithms has high realtime performance and recognition rate.