本文已被:浏览 1056次 下载 2361次
Received:May 25, 2020 Revised:June 16, 2020
Received:May 25, 2020 Revised:June 16, 2020
中文摘要: 随着数据的不断积累, 数据存储系统已经由单节点、小规模的文件系统演变成支持海量数据存储的分布式文件系统. 分布式文件系统中的大规模数据如何能够面向用户提供简洁、高效和稳定的数据上传下载服务成为了广泛研究的问题. 为此, 本文提出了基于FTP协议的数据服务端的解决方案, 该方案包括利用缓冲区、协程、多线程等技术优化传输模型和传输算法, 使得数据流能进行高速传输. 同时, 配合Keepalived (开源的系统高可用解决方案)与LVS实现了高可用、高并发以及良好的拓展性, 在FTP协议之上修改使得服务端支持动态验证、用户隔离并兼容多种编码格式. 在实际的运行中, 以中国科技云iHarbor存储系统为例, 验证了本文提出方案的有效性、可靠性以及高性能的数据传输效果, 可以充分满足用户的需求.
Abstract:With the continuous accumulation of data, the data storage system has evolved from a single-node, small-scale file system to a distributed file system that supports massive data storage. How to provide simple, efficient and stable data uploading and downloading service for users has become an extensive research problem in distributed file system. To this end, we propose a server-side solution based on FTP protocol, which includes the use of buffer, coroutine, multi-process, and other technologies to improve the algorithms and optimize the transmission model, so that the data flow can be transferred rapidly. The proposed solution achieves high availability, high concurrency, and good scalability with the cooperation of Keepalived and LVS. Modifications to the FTP protocol makes the server support dynamic validation and compatible with multiple encoding formats. In the actual operation, taking storage system of China Science and Technology Cloud, i.e. iHarbor, as an example, the effectiveness, reliability, and performance of data transmission of the proposed solution are verified, which can fully meet the needs of users.
文章编号: 中图分类号: 文献标志码:
基金项目:中国科学院A类战略性先导科技专项(XDA19000000)
引用文本:
李云,张海明.基于FTP协议的数据服务解决方案.计算机系统应用,2021,30(1):63-69
LI Yun,ZHANG Hai-Ming.Data Service Solution Based on FTP Protocol.COMPUTER SYSTEMS APPLICATIONS,2021,30(1):63-69
李云,张海明.基于FTP协议的数据服务解决方案.计算机系统应用,2021,30(1):63-69
LI Yun,ZHANG Hai-Ming.Data Service Solution Based on FTP Protocol.COMPUTER SYSTEMS APPLICATIONS,2021,30(1):63-69