Adaptive independence assumption Non-autoregressive Transformer for speech recognition

The non-autoregressive Transformer based end-to-end automatic speech recognition model has a faster decoding speed compared with traditional models such as autoregressive Transformer, however, the non-autoregressive decoding method and independence assumption lead to the degradation of speech recognition result accuracy. To address this problem, a non-autoregressive Transformer Chinese speech recognition model with adaptive independence assumption and speech representation fusion is proposed. During training, the problem of partially missing semantic information in the input frame of the decoder is improved by attention fusion of the representation vectors; during decoding, adaptive independence assumption is used to solve the problem of conditional independence of the output characters brought by the independence assumption of the non-autoregressive model. Finally, iterative beam search is used to perform ranking search decoding of multiple targets to solve the inapplicability problem of the beam search algorithm in the proposed model. The experimental results on the Chinese dataset AISHELL-1 show that the real time factor of the model reaches 0.005 and the character error rate is 8.8%, which is 20% lower than the non-autoregressive Transformer baseline model, ensuring a higher recognition speed while significantly reducing the error rate, showing the advanced model performance.