With the explosive growth of mobile applications, how to carry out UI automation testing efficiently and correctly becomes an important issue. Most of the traditional automated methods require developers to write test scripts manually, and another high-level testing method called "recording and playback" does not has capability of crossing devices. In addition, existing assertion mechanisms are generally not capable of describing the UI semantics completely. Due to those problems, this paper presents a new recording and playback method which is capable of crossing devices and describing the UI semantics. This method uses the widget path to precisely locate the widgets and employs the cross-device UI adaptive method to improve the capability of device-crossing. Furthermore, this study proposes two new assertion mechanisms to support UI semantics which check on number sorting and pictures. In addition, this study builds a prototype framework called RRF according to the proposed methods, and the experimental results show that RRF has a higher success rate of playback than other automated test tools.