With the development of deep learning technology, the performance of natural scene text detection has been significantly improved. Nonetheless, two main challenges still exist: the first problem is the trade-off between speed and accuracy, and the second one is to model the arbitrary-shaped text instance. In this study, we propose a segmentation-based method to tackle arbitrary-shaped text detection efficiently and accurately. Specifically, we use a low computational-cost segmentation head and efficient post-processing. The segmentation head is made up of Feature Pyramid Enhancement Module (FPEM) and Feature Fusion Module (FFM). FPEM can introduce multi-level information to guide the better segmentation. FFM can integrate the features given by the FPEMs of different depths into a final feature for segmentation. We use a Differentiable Binarization (DB) module, which can perform the binarization process in a segmentation network. Optimized along with a DB module, a segmentation network can adaptively set the thresholds for binarization, which not only simplifies the post-processing but also enhances the performance of text detection. On the standard datasets ICDAR2015 and Total-Text, the method proposed in this study uses a lightweight backbone network such as ResNet18 to achieve comparable results in terms of speed and accuracy.