Gait recognition is an emerging biometric technology, which can be widely used in criminal security, epidemic transmission chain tracking, etc. The essence of this technology is to identify people’s identity, age, gender and other biological attributes through their human body shape and walking posture. Compared with other biometric technologies, gait recognition has significant advantages such as long distance, full view, no perception, and anti-counterfeiting. In this study, we design a cross-view gait tracking system for multiple people and multiple biological attributes. The system fully considers the impact of covariates (such as multiple people, cross view and clothing change) on gait recognition accuracy in real application scenarios. It extracts the gait information of pedestrians from complex environments to accurately analyze their biological attributes such as identity, age, and gender through a more robust algorithm design. The experimental results show that the accuracy of the deep learning-based gait recognition algorithm model in this system can reach 88.0% in the case of cross view and multiple walking states and 94.8% in the case of multiple views for gender classification. The average age error of age estimation is about 7.92 years with a standard deviation of about 8.11. These results are better than those of recent algorithms in related fields and reach a relatively leading level. At a low development cost, the system is oriented to application scenarios and supports real-time gait detection.