MFUIE: A Fake News Detection Model Based on Multimodal Features and User Information Enhancement

INTRODUCTION: Deep learning algorithms have advantages in extracting key features for detecting fake news. However, the existing multi-modal fake news detection models only fuse the visual and textual features after the encoder, failing to effectively utilize the multi-modal contextual relationships and resulting in insufficient feature fusion. Moreover, most fake news detection algorithms focus on mining news content and overlook the users' preferences whether to spread fake news. OBJECTIVES: The model uses the multi-modal context relationship when extracting model features, and combines with user features to assist in mining multi-modal information to improve the performance of fake news detection. METHODS: A fake news detection model called MFUIE (Multimodal Feature and User Information Enhancement) is proposed, which utilizes multi-modal features and user information enhancement. Firstly, for news content, we utilize the pre-trained language model BERT to encode sentences. At the same time, we use the Swin Transformer model as the main framework and introduce textual features during the early visual feature encoding to enhance semantic interactions. Additionally, we employ InceptionNetV3 as the image pattern analyser. Secondly, for user's historical posts, we use the same model as the news text to encode them, and introduce GAT (Graph Attention Network) to enhance information interaction between post nodes, capturing user-specific features. Finally, we fuse the obtained user features with the multi-modal features and validate the performance of the model. RESULTS: The proposed model's performance is compared with those of existing methods. MFUIE model achieves an accuracy of 0.926 and 0.935 on the Weibo dataset and Weibo-21 dataset, respectively. F1 on Weibo is 0.926, 0.017 greater than SOAT model BRM; while F1 on Weibo-21 is 0.935, 0.009 greater than that of BRM. CONCLUSION: Experimental results demonstrate that MFUIE can improve the fake news recognition in some degree.