Amid the widespread application of face recognition technology for authentication, various attack methods against face recognition systems have emerged over time. An identity authentication system based on lip reading recognition is proposed to cope with such security issues. It requires users to read the verification code during face recognition for authentication. The system not only checks the face, but recognizes the content of the speech through the lip reading recognition technology to compare it with the verification code. The user’s identity can be authenticated only when both the two links are passed. Finally, an identity authentication system based on lip reading recognition is designed, mainly including front-end, gateway, and back-end.