Abstract:In the reinforcement learning method for the continuous integration test case prioritization (CITCP), the agent rewards the test cases to realize the adjustment of test case prioritization strategy, and thus they can meet the needs of frequent iteration and rapid feedback in continuous integration testing. The agent usually only rewards the failure test cases. However, in the actual industrial processes, the continuous integration testing features high-frequency integration and low-failure-rate tests, which poses a new challenge to the actual application of CITCP. Low-failure-rate tests can be understood as a sparse number of failure test cases, which can lead to the sparsity of reward objects in reinforcement learning and bring about the sparse reward problem. In this study, a reward object selection strategy is proposed to solve the sparse reward problem. With the failure test cases rewarded, passing test cases similar to failure test cases are selected to be rewarded, and thus the number of reward objects increases. Specifically, the similarity measure method for test cases is designed with the feature vector representation of historical execution information sequences and duration time. Then, the passing test cases similar to the failure test cases are selected to be rewarded through the similarity measure. The experiments are conducted in six industrial data sets, and the results show that the similarity-based reward object selection strategy can effectively solve the sparse reward problem by increasing the reward objects and further improve the quality of reinforcement learning-based CITCP.