The rectangular form is a basic shape in the gray-scale image, and it’s an important applications in image analysis. In order to effectively detect rectangular form in image, a rectangular form filter is proposed. Firstly, the structuring elements of the rectangular form filter are defined, and then two kinds of filtering mode are defined: order filtering and mean filtering. The experimental results show that our filter can effectively detecte the rectangular form specified the size and direction from binary images, binary images corruted by noise and gray-scale images.