sis 18: e71

Research Article

Sentence classification based on the concept kernel attention mechanism

Download205 downloads
  • @ARTICLE{10.4108/eai.17-5-2022.173980,
        author={Hui Li and Guimin Huang and Yiqun Li and Xiaowei Zhang and Yabing Wang},
        title={Sentence classification based on the concept kernel attention mechanism},
        journal={EAI Endorsed Transactions on Scalable Information Systems: Online First},
        volume={},
        number={},
        publisher={EAI},
        journal_a={SIS},
        year={2022},
        month={5},
        keywords={Sentence classification, text conceptualization, concept knowledge base, attention mechanism, concept embeddings},
        doi={10.4108/eai.17-5-2022.173980}
    }
    
  • Hui Li
    Guimin Huang
    Yiqun Li
    Xiaowei Zhang
    Yabing Wang
    Year: 2022
    Sentence classification based on the concept kernel attention mechanism
    SIS
    EAI
    DOI: 10.4108/eai.17-5-2022.173980
Hui Li1, Guimin Huang1,*, Yiqun Li1, Xiaowei Zhang1, Yabing Wang1
  • 1: Guangxi Key Laboratory of Image and Graphic Intelligent Processing, School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, China
*Contact email: sendhuang@126.com

Abstract

Sentence classification is important for data mining and information security. Recently, researchers have paid increasing attention to applying conceptual knowledge to assist in sentence classification. Most existing approaches enhance classification by finding word-related concepts in external knowledge bases and incorporating them into sentence representations. However, this approach assumes that all concepts are equally important, which is not helpful for distinguishing the categories of the sentence. In addition, this approach may also introduce noisy concepts, resulting in lower classification performance. To measure the importance of the concepts for the text, we propose the Concept Kernel Attention Network (CKAN). It not only introduces concept information into the deep neural network but also contains two attention mechanisms to assign weights to concepts. The attention mechanisms are the text-to-concept attention mechanism (TCAM) and the entity-to-concept attention mechanism (ECAM). These attention mechanisms limit the importance of noisy concepts as well as contextually irrelevant concepts and assign more weights to concepts that are important for classification. Meanwhile, we combine the relevance of concepts and entities to encode multi-word concepts to reduce the impact of the inaccurate representation of multi-word concepts for classification. We tested our model on five public text classification datasets. Comparison experiments with strong baselines and ablation experiments demonstrate the effectiveness of CKAN.