Abstract:A dynamic data partition system based on node load mainly considers the load of CPU, memory, and bandwidth of nodes. It first uses the quadratic smoothing method to predict the load of nodes, then combines AHP and entropy index weight to get the processing capacity of each node, and finally dynamically adjusts the load balance of the system for different application scenarios to improve the response speed of applications. It includes the modules of load monitoring and collection, prediction, data pre-partition, and data migration. Given the heterogeneity of node resources in a distributed environment, it aims to reduce the data transmission between nodes in the process of data analysis and calculation, make full use of node computing resources, and improve the parallel computing speed of application analysis through load balancing. Therefore, this study proposes a dynamic partition data mechanism based on node load to improve the system load balance and application response speed and assist the relevant staff in making the decision. This study combines data analysis application scenarios integrating Spark and Elasticsearch for testing.