Research Article
A Comprehensive Evaluation Method for KG-Augmented Large Language Models
@INPROCEEDINGS{10.1007/978-3-031-78806-2_7, author={Xingyu Chen and Ligang Dong and Meng Han}, title={A Comprehensive Evaluation Method for KG-Augmented Large Language Models}, proceedings={Smart Grid and Innovative Frontiers in Telecommunications. 8th EAI International Conference, EAI SmartGIFT 2024a, Santa Clara, United States, March 23-24, 2024, Proceedings}, proceedings_a={SMARTGIFT}, year={2025}, month={1}, keywords={Large Language Model Knowledge Graph Augmented}, doi={10.1007/978-3-031-78806-2_7} }- Xingyu Chen
Ligang Dong
Meng Han
Year: 2025
A Comprehensive Evaluation Method for KG-Augmented Large Language Models
SMARTGIFT
Springer
DOI: 10.1007/978-3-031-78806-2_7
Abstract
Integrating factual information from knowledge graphs (KGs) into large language models (LLMs) has emerged as a promising approach to mitigate hallucination issues inherent in LLMs. This augmentation not only addresses the problem of generating inaccurate or fictional content but also offers avenues for tailoring LLMs to specific domains. Despite the potential benefits, the current body of research lacks a thorough examination of how KG augmentation influences various large language models.
This paper introduces a comprehensive evaluation method specifically designed for assessing the performance of KG-augmented LLMs. By evaluating these frameworks across multiple dimensions, the proposed method aims to provide a nuanced understanding of the strengths and limitations associated with integrating KGs into LLM-based question-answering systems. The systematic evaluation is expected to offer valuable insights, guiding future research endeavors and facilitating enhancements in this emerging field. This approach contributes to advancing the integration of KGs with LLMs and fostering the development of more robust and context-aware language models tailored to specific knowledge domains.
