Distributed Object Storage System for Scientific Research
CSTR:
Author:
Affiliation:

Clc Number:

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    With the development of scientific research, there is a massive increase in scientific research data. PB-level scientific research data requires efficient and stable storage systems. The traditional data storage scheme has problems such as poor resource utilization, low cluster expansion performance, and unfriendly user interface operation, which seriously limit the effective use of data. Relying on the Big Data Project of the Chinese Academy of Sciences, we design and implement an efficient storage system i-Harbor. Its core architecture is based on object storage system, using open-sourced Ceph distributed system and MongoDB database as the storage carrier of object data and metadata. The data interface is designed on the basis of HTTP API and FTP. To improve the platform disaster tolerance and security, we use Multiple Copies and Erasure Coding technology to eliminate single node of failure. Meanwhile we locate the real-time platform parameters and faults by Zabbix cluster monitoring system. Based on the distribution characteristics, the cluster can add storage nodes at will, which improves the platform’s scalability.

    Reference
    Related
    Cited by
Get Citation

王锦涛,张海明.面向科研领域的分布式对象存储系统.计算机系统应用,2020,29(7):82-88

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:November 20,2019
  • Revised:December 16,2019
  • Adopted:
  • Online: July 04,2020
  • Published: July 15,2020
Article QR Code
You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063