登录 注册

A Multi-Source Framework for Relational Validation of Large Language 模型 (Model)s Using Expert-Curated Encyclopedic Sources
A Multi-Source Framework for Relational Validation of Large Language Models Using Expert-Curated Encyclopedic Sources

🔗 访问原文
🔗 Access Paper

📝 摘要
Abstract

This paper introduces a novel, multi-source framework for the relational validation of Large Language Models (LLMs). While existing benchmarks have demonstrated LLMs' proficiency at factual recall, their ability to understand and reproduce the intricate web of relationships that defines a domain's conceptual structure remains largely unexplored. Our three-layer analytical framework provides a scalable and robust methodology for assessing the depth of an LLM's knowledge across diverse academic domains. By comparing LLM-generated knowledge graphs to expert-curated encyclopedias, we reveal a consistent and significant ``relational deficit'': LLMs recognize domain-specific concepts but consistently fail to reproduce their relational structure. Our findings highlight the need for more sophisticated evaluation metrics that go beyond simple accuracy and assess the relational integrity of an LLM's knowledge. We demonstrate that this deficit is highly domain-dependent, with performance varying significantly across ten specialized encyclopedias spanning sociology, political science, philosophy, and other fields. The cases of complete relational failure in the most specialized domains are particularly revealing, suggesting that the LLM's internal knowledge representation is not aligned with the conceptual structures of these fields. This has significant implications for the deployment of LLMs in high-stakes applications that require a deep, nuanced understanding of domain-specific knowledge.

📊 文章统计
Article Statistics

基础数据
Basic Stats

199 浏览
Views
0 下载
Downloads
28 引用
Citations

引用趋势
Citation Trend

阅读国家分布
Country Distribution

阅读机构分布
Institution Distribution

月度浏览趋势
Monthly Views

相关关键词
Related Keywords

影响因子分析
Impact Analysis

4.50 综合评分
Overall Score
引用影响力
Citation Impact
浏览热度
View Popularity
下载频次
Download Frequency

📄 相关文章
Related Articles

海洋智能分析Ocean AI Analysis

正在分析中,请稍候…Analyzing, please wait…
🌊