本文已被:浏览 934次 下载 1866次
Received:September 14, 2021 Revised:October 14, 2021
Received:September 14, 2021 Revised:October 14, 2021
中文摘要: 语音可懂度增强是一种在嘈杂环境中再现清晰语音的感知增强技术. 许多研究通过说话风格转换(SSC)来增强语音可懂度, 这种方法仅依靠伦巴第效应, 因此在强噪声干扰下效果不佳. SSC还利用简单的线性变换对基频(F0)的转换进行建模, 并且只映射很少维的梅尔倒谱系数(MCEPs). 因为F0和MCEPs是语音的两个重要特征, 对这些特征进行充分的建模是非常必要的. 因此本文进行了一个创新性研究即通过连续小波变换(CWT)将F0分解为10维来描述不同时间尺度的语音, 以实现F0的有效转换, 而且使用20维表示MCEPs实现MCEPs的转换. 除此之外, 还利用iMetricGAN网络来优化强噪声中的语音可懂度指标. 实验结果表明, 提出的基于CycleGAN使用CWT和iMetricGAN的非平行语音风格转换方法(NS-CiC)在客观和主观评价上均显著提高了强噪声环境下的语音可懂度.
中文关键词: 深度学习 可懂度增强 连续小波变换 iMetricGAN CycleGAN
Abstract:Speech intelligibility enhancement is a perceptual enhancement technique for clean speech reproduced in noisy environments. Speaking style conversion (SSC) is used in many studies to achieve speech intelligibility, which relies solely on the Lombard effect and thus demonstrates poor performance with strong noise interference. In addition, the SSC method models the conversion of fundamental frequency (F0) with a straight forward linear transform and only maps Mel-frequency cepstral coefficients (MFCCs) with few dimensions. As F0 and MFCCs are critical aspects of hierarchical intonation, adequate modeling of these features is essential. Therefore, we use the continuous wavelet transform (CWT) to decompose F0 into ten dimensions to describe speech at different time scales for effective F0 conversion and represent MFCCs with 20 dimensions for MFCC conversion. Furthermore, we utilize an iMetricGAN to optimize speech intelligibility metrics in strong noise. The experimental results show that in objective and subjective evaluations, the proposed non-parallel speech style conversion method using CWT and iMetricGAN based on CycleGAN (NS-CiC) significantly increases speech intelligibility in robust noise environments.
keywords: deep learning intelligibility?enhancement continuous?wavelet?transform (CWT) iMetricGAN CycleGAN
文章编号: 中图分类号: 文献标志码:
基金项目:国家重点研发计划(1502-211100026)
引用文本:
肖晶,刘佳奇,李登实,赵兰馨,王前瑞.基于CycleGAN的语音可懂度关键技术.计算机系统应用,2022,31(6):1-9
XIAO Jing,LIU Jia-Qi,LI Deng-Shi,ZHAO Lan-Xin,WANG Qian-Rui.Key Technologies of Speech Intelligibility Based on CycleGAN.COMPUTER SYSTEMS APPLICATIONS,2022,31(6):1-9
肖晶,刘佳奇,李登实,赵兰馨,王前瑞.基于CycleGAN的语音可懂度关键技术.计算机系统应用,2022,31(6):1-9
XIAO Jing,LIU Jia-Qi,LI Deng-Shi,ZHAO Lan-Xin,WANG Qian-Rui.Key Technologies of Speech Intelligibility Based on CycleGAN.COMPUTER SYSTEMS APPLICATIONS,2022,31(6):1-9