With the lower level of automation and high labor costs in the present fabric industry, it is inevitable that the robot will replace human to work. The article presents the robot fabirc scraping system based on the Kinect. First of all, target fabric image is obtained by making use of Kinect. Then, target fabric can be found out by CamShift algorithm and the center of mass is gotten. Finally, the coordinates can be transformed into motion coefficient with robot kinematics to make robot scraping fabric come ture. The experiment results imply the method which can get three-dimensional coordinates of target fabric quickly has more advantages than visual robot.