Abstract:To further improve the performance of data de-duplication system, the paper brings out a de-duplication system based on data blocks’ suffix array and longest common prefix for the block size optimization. The method first chunks input data into segments with a first size, then identifies the segments and create the indexes queue for the identifiers, and then create the suffix array and the longest common prefix structure from the indexes, next determines a second optimize size based on repeated indexes sequences and non-repeated indexes, finally chunks the input data into segments based on super chunks and saves the compression result. The result shows that it achieves a better compression ratio and object reconstruction for the given input data comparing the original fixed chunk segment size.