Cataract is an ocular disease that mainly causes visual impairment and blindness, and early intervention and cataract surgery are the primary ways of improving the vision and the life quality of cataract patients. Anterior segment optical coherence tomography (AS-OCT) is a new type of ophthalmic image featuring non-contact, high resolution, and quick examination. In clinical practice, ophthalmologists have gradually used AS-OCT images to diagnose ophthalmic diseases such as glaucoma. However, none of the previous works have focused on automatic cortical cataract (CC) classification with such images. For this reason, this study proposes an automatic CC classification framework based on AS-OCT images, and it is composed of image preprocessing, feature extraction, feature screening, and classification. First, the reflective region removal and contrast enhancement methods are employed for image preprocessing. Next, 22 features are extracted from the cortical region by the gray level co-occurrence matrix (GLCM), grey level size zone matrix (GLSZM), and neighborhood gray-tone difference matrix (NGTDM) methods. Then, the Spearman correlation coefficient method is used to analyze the importance of the extracted features and screen out redundant ones. Finally, the linear support vector machine (linear-SVM) method is utilized for classification. The experimental results on a clinical AS-OCT image dataset show that the proposed CC classification framework achieves 86.04% accuracy, an 86.18% recall rate, 88.27% precision, and 86.35% F1-score respectively and obtains performance comparable to that of the advanced deep learning-based algorithm, indicating that it has the potential to be used as a tool to assist ophthalmologists in clinical CC diagnosis.