The roadside perception algorithm is integrated with the on-board perception algorithm to achieve over-the-horizon perception. The performance of the perception algorithm based on deep learning depends on the quality of the point cloud annotation of lidar which is harder than the annotation of 2D images because it takes longer time and calls for much manpower. In addition, existing perception algorithms based mainly on the on-board lidar. In this study, we proposes a perception algorithm based on the feature clustering of roadside lidar grids. This algorithm rasterizes the point cloud of roadside lidar and extract the features, then learn the primary perception information of the grids by creating a deep learning model for clustering on this basis. We also simulate the point cloud of roadside lidar via a simulation platform, and studies the application of the hybrid data set in training perception algorithm, which is fine-tuned by the pre-training model of simulation data. Experimental results show that the proposed perception algorithm is reliable with real-time service. Besides, simulating the point cloud of roadside lidar helps with the training of this algorithm and reduces its dependence on annotation, improving its performance.