###
计算机系统应用:2018,27(8):214-218
本文二维码信息
码上扫一扫!
基于Spark的分布式数字信号处理算法库设计
乔昕1,2, 刘峰1,2, 于碧辉1,2
(1.中国科学院大学, 北京 100049;2.
中国科学院 沈阳计算技术研究所, 沈阳 110168)
Design of Distributed Digital Signal Processing Algorithm Library Based on Spark
QIAO Xin1,2, LIU Feng1,2, YU Bi-Hui1,2
(1.University of Chinese Academy of Sciences, Beijing 100049, China;2.
Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang 110168, China)
摘要
图/表
参考文献
相似文献
本文已被:浏览 118次   下载 116
投稿时间:2018-01-02    修订日期:2018-01-23
中文摘要: 传统的基于DSP与FPGA的数字信号处理技术更加适用于实时信号处理,且受到数据规模和频率分辨率的限制,使得其不适于进行大规模数据下的离线式数据处理、分析与挖掘的应用.目前工业大数据分析平台可以采用Spark作为实时信号处理和离线信号处理加速的计算引擎,但该分析平台缺少适用于分布式并行计算引擎的数字信号处理等数学计算的解决方案.基于此,本文提出了基于Spark的分布式数字信号处理算法库,为面向分析的工业大数据应用场景提供支撑.本文介绍了该算法库的架构设计,并以FFT算法和DFT算法为例介绍了传统数字信号处理算法在Spark下的分布式实现,最后对算法库进行了正确性测试和性能分析.结果表明该算法库能够正确完成数字信号处理的功能,同时可以满足工业大数据分析平台对于大规模数据集进行数字信号处理的需求.
Abstract:The traditional DSP and FPGA-based digital signal processing technology is more suitable for real-time signal processing. It is limited by the size and frequency resolution of the data. So it unsuitable for applications of off-line data processing, data analysis and data mining under large-scale data. Currently the industrial big data analytics platforms can use Spark as a computational engine for real-time signal processing and off-line signal processing acceleration. However, there is a lack of mathematical solutions such as digital signal processing for distributed parallel computing engines. Consequently, this paper presents a library of distributed digital signal processing algorithms based on Spark, which provides a support for the analysis of industrial big data application scenarios. This paper describes the architecture of the algorithm library design and takes the FFT algorithm and DFT algorithm as examples to introduce the distributed implementation of the traditional digital signal algorithm in Spark. Finally, this paper persents a test and analysis for this algorithm library. The results show that the algorithm library can correctly accomplish the function of digital signal processing and it can fulfill the industrial big data analysis platform for large-scale data sets for digital signal processing needs.
文章编号:     中图分类号:    文献标志码:
基金项目:
引用文本:
乔昕,刘峰,于碧辉.基于Spark的分布式数字信号处理算法库设计.计算机系统应用,2018,27(8):214-218
QIAO Xin,LIU Feng,YU Bi-Hui.Design of Distributed Digital Signal Processing Algorithm Library Based on Spark.COMPUTER SYSTEMS APPLICATIONS,2018,27(8):214-218

用微信扫一扫

用微信扫一扫