Abstract:In recent years, deep learning, as a hotspot of common concern in academia and industry, has made great progress and achieved remarkable achievements in computer vision, speech recognition and other fields. It is divided into two stages:training and inferencing. In practical application, the main concern is the inferencing stage. The process of deep learning inferecing is accompanied by a huge amount of computation, and more and more attention has been paid to using distributed system to improve its computing speed. However, the construction of distributed deep learning inferencing system is faced with the challenges such as rapid updating and iteration of deep learning accelerators, complex of applications and computing tasks. The information management mechanism proposed in this study is used to collect and process all kinds of information in the distributed system, and the rules of collection and processing are highly customizable and flexible. It also provides a universal RESTful API data access interface to support the flexible compatibility of various hardware and the dynamic adjustment ability of task scheduling strategy in the deep learning inferencing system. Finally, we verified the function of the mechanism through an example and analysed the experimental results.