Multi-Stage Speech Bandwidth Extension with Flexible Sampling Rates Control

National Engineering Research Center of Speech and Language Information Processing
University of Science and Technology of China

Abstract

The majority of existing speech bandwidth extension (BWE) methods operate under the constraint of fixed input and target sampling rates, which limits their flexibility in practical applications. In this paper, we propose a multi-stage speech BWE model named MS-BWE, which can handle a set of input and target sampling rate pairs and achieves a flexible extension of frequency bandwidth. The proposed MS-BWE model comprises a cascade of BWE blocks, progressively painting the speech frequency bands stage by stage. Each BWE block features a dual-stream architecture to realize amplitude and phase extension, respectively. The teacher-forcing strategy is employed to mitigate the discrepancy between training and inference. Experimental results demonstrate that our proposed MS-BWE is comparable to state-of-the-art baseline methods in speech quality. Regarding generation efficiency, the one-stage generation of MS-BWE can achieve over one thousand times real-time on GPU and about sixty times on CPU.


I. Audio Samples with Target Sampling Rate of 48kHz


8 kHz to 48 kHz


Wideband Sinc NU-Wave 2 UDM+ mdctGAN AP-BWE MS-BWE (Four Stages)

12 kHz to 48 kHz


Wideband Sinc NU-Wave 2 UDM+ mdctGAN AP-BWE MS-BWE (Three Stages)

16 kHz to 48 kHz


Wideband Sinc NU-Wave 2 UDM+ mdctGAN AP-BWE MS-BWE (Two Stages)

24 kHz to 48 kHz


Wideband Sinc NU-Wave 2 UDM+ mdctGAN AP-BWE MS-BWE (One Stage)

II. Audio Samples with Flexible Sampling Rates


8 kHz to 12 kHz


Wideband Sinc mdctGAN AP-BWE MS-BWE (One Stage)

8 kHz to 16 kHz


Wideband Sinc mdctGAN AP-BWE MS-BWE (Two Stages)

8 kHz to 24 kHz


Wideband Sinc mdctGAN AP-BWE MS-BWE (Three Stages)

12 kHz to 16 kHz


Wideband Sinc mdctGAN AP-BWE MS-BWE (One Stage)

12 kHz to 24 kHz


Wideband Sinc mdctGAN AP-BWE MS-BWE (Two Stages)

16 kHz to 24 kHz


Wideband Sinc mdctGAN AP-BWE MS-BWE (One Stage)

III. Audio Samples of Different Training Strategies


8 kHz to 48 kHz (Four-Stage Generation)


Wideband Sinc Never Sampling Always Sampling Scheduled Sampling

12 kHz to 48 kHz (Three-Stage Generation)


Wideband Sinc Never Sampling Always Sampling Scheduled Sampling

16 kHz to 48 kHz (Two-Stage Generation)


Wideband Sinc Never Sampling Always Sampling Scheduled Sampling

24 kHz to 48 kHz (One-Stage Generation)


Wideband Sinc Never Sampling Always Sampling Scheduled Sampling