Research Article
Sentence classification based on the concept kernel attention mechanism
@ARTICLE{10.4108/eai.17-5-2022.173980, author={Hui Li and Guimin Huang and Yiqun Li and Xiaowei Zhang and Yabing Wang}, title={Sentence classification based on the concept kernel attention mechanism}, journal={EAI Endorsed Transactions on Scalable Information Systems}, volume={10}, number={1}, publisher={EAI}, journal_a={SIS}, year={2022}, month={5}, keywords={Sentence classification, text conceptualization, concept knowledge base, attention mechanism, concept embeddings}, doi={10.4108/eai.17-5-2022.173980} }
- Hui Li
Guimin Huang
Yiqun Li
Xiaowei Zhang
Yabing Wang
Year: 2022
Sentence classification based on the concept kernel attention mechanism
SIS
EAI
DOI: 10.4108/eai.17-5-2022.173980
Abstract
Sentence classification is important for data mining and information security. Recently, researchers have paid increasing attention to applying conceptual knowledge to assist in sentence classification. Most existing approaches enhance classification by finding word-related concepts in external knowledge bases and incorporating them into sentence representations. However, this approach assumes that all concepts are equally important, which is not helpful for distinguishing the categories of the sentence. In addition, this approach may also introduce noisy concepts, resulting in lower classification performance. To measure the importance of the concepts for the text, we propose the Concept Kernel Attention Network (CKAN). It not only introduces concept information into the deep neural network but also contains two attention mechanisms to assign weights to concepts. The attention mechanisms are the text-to-concept attention mechanism (TCAM) and the entity-to-concept attention mechanism (ECAM). These attention mechanisms limit the importance of noisy concepts as well as contextually irrelevant concepts and assign more weights to concepts that are important for classification. Meanwhile, we combine the relevance of concepts and entities to encode multi-word concepts to reduce the impact of the inaccurate representation of multi-word concepts for classification. We tested our model on five public text classification datasets. Comparison experiments with strong baselines and ablation experiments demonstrate the effectiveness of CKAN.
Copyright © 2022 Hui Li et al., licensed to EAI. This is an open access article distributed under the terms of the Creative Commons Attribution license, which permits unlimited use, distribution and reproduction in any medium so long as the original work is properly cited.