hl7-fhir-r4-1656978e·2 events·first seen Aliases: HL7 FHIR R4, HL7 FHIR
A new arXiv paper evaluates three open-source models (Qwen2.5 7B, Llama 3.1 8B, Gemma2 9B) on schema compliance with ICD-10, CPT, and HL7 FHIR standards across 960 clinical scenario-model pairs. Baseline compliance ranged from 85.9–91.6%, with 96% of failures being representation-level format violations rather than clinical reasoning errors. A closed-loop validation-repair framework raised overall compliance to 99.0%, with most errors resolving in one or two iterations, suggesting this system-level approach is a viable safeguard for healthcare EHR integration.
The paper introduces a pipeline for converting unstructured clinical text into HL7 FHIR R4 bundles, enabling evaluation of LLMs in realistic electronic health record settings. Applied to the MedCaseReasoning dataset, it produces MedCase-Structured, a synthetic benchmark achieving valid FHIR generation for 82.5% of cases. Key finding: LLMs show consistently lower diagnostic accuracy on structured FHIR inputs compared to plain text, underscoring the gap between standard benchmarks and real-world clinical deployment conditions.