梅侠峰, 吴晓鸰, 吴杰文, 凌捷, HoonHeo. 基于RoBERTa-BiSRU++-AT的微博谣言早期检测模型[J]. 微电子学与计算机, 2022, 39(2): 34-42. DOI: 10.19304/J.ISSN1000-7180.2021.0722
引用本文: 梅侠峰, 吴晓鸰, 吴杰文, 凌捷, HoonHeo. 基于RoBERTa-BiSRU++-AT的微博谣言早期检测模型[J]. 微电子学与计算机, 2022, 39(2): 34-42. DOI: 10.19304/J.ISSN1000-7180.2021.0722
MEI Xiafeng, WU Xiaoling, WU Jiewen, LING Jie, Hoon Heo. Early detection of microblog rumors based on RoBERTa-BiSRU++-AT model[J]. Microelectronics & Computer, 2022, 39(2): 34-42. DOI: 10.19304/J.ISSN1000-7180.2021.0722
Citation: MEI Xiafeng, WU Xiaoling, WU Jiewen, LING Jie, Hoon Heo. Early detection of microblog rumors based on RoBERTa-BiSRU++-AT model[J]. Microelectronics & Computer, 2022, 39(2): 34-42. DOI: 10.19304/J.ISSN1000-7180.2021.0722

基于RoBERTa-BiSRU++-AT的微博谣言早期检测模型

Early detection of microblog rumors based on RoBERTa-BiSRU++-AT model

  • 摘要: 针对现有微博谣言检测算法在实用场景中存在滞后性,以及传统深度学习模型特征提取能力不足等问题,提出了基于RoBERTa-BiSRU++-AT的微博谣言早期检测模型,仅使用微博原始文本作为模型输入,不包含任何转发和评论信息或者其他相关特征信息.采用RoBERTa预训练模型学习当前词在特定上下文语境中的动态含义,解决静态词向量无法表示多义词的问题,提升词的语义表征能力;通过双向内置注意力简单循环单元(Simple Recurrent Unit with Built-in Self-Attenttion)进行深层语义特征抽取,自注意力机制可以捕获句子内部词与词之间的依赖关系, 得到更为全面的高维特征;引入软注意力机制计算不同词对分类结果的重要程度,赋予模型聚焦关键特征的能力,解决输出特征对分类结果影响力一致的问题;得到的软注意力特征经Softmax层计算得到分类概率,取概率最大值对应标签为分类结果.在公开的中文微博谣言数据集进行实验,实验结果表明,本文所提出的基于RoBERTa-BiSRU++-AT的模型F1分数达到了98.16%,高于实验对比的其他微博谣言检测算法,证明该模型对微博文本谣言具有更好的早期识别能力.

     

    Abstract: In order to solve the problem of lag in the actual application of the existing microblog rumor detection algorithms and the insufficient feature extraction ability of traditional deep learning model, this paper proposes an early detection model of microblog rumors based on RoBERTa-BiSRU++-AT. The original text is used as the input, which does not contain forwarding and comment or other relevant information. The RoBERTa pre-trained model assigns dynamic and contextual semantics to words, which sloves the polysemy and improves the semantic representation ability of words. Furthermore, the bidirectional simple recurrent unit with built-in self-attention extracts the deep semantic features, so self-attention can learn the dependencies between words, which can provide comprehensive high-dimensional features. The soft attention mechanism is introduced to calculate the weight of words on classification result, which can solve the problem that different features have the same influence on classification result, and the model can focus on key features. The classification probability is calculated by Softmax layer, and the label corresponding to the maximum probability is taken as the result. Comparative experiment was carried out on the Chinese microblog rumors dataset. The experimental results prove that the F1 index of RoBERTa-BiSRU++-AT is 98.16%, higher than other microblog rumor detection algorithms, which proves that the proposed model has better early recognition ability for microblog text rumor.

     

/

返回文章
返回