FAIR Principles

Comparative Study: Semantic Metadata vs. Unstructured Web Retrieval for Agentic Data Discovery

This paper evaluates whether LLM-based agents still need structured semantic metadata (e.g., schema.org) for data retrieval, comparing a Baseline Agent searching open-web documents against a Semantic Agent leveraging 90 million schema.org-annotated datasets. Using an LLM-as-a-judge pipeline aligned to FAIR principles, the Semantic Agent achieves 65.7% higher overall precision in retrieving FAIR-compliant datasets, while the Baseline Agent answers 40% more questions but frequently returns prose-heavy or portal landing pages instead of actionable data. The study concludes that structured semantic ecosystems remain essential for reliable, execution-oriented agentic workflows despite LLMs' broad unstructured retrieval capabilities.