paper

Clinically Grounded Privacy Evaluation of Medical LMs

paperactiveprovisionalclinically-grounded-privacy-evaluation-of-medical-lms-355bdf58·1 events·first seen 8d ago

Aliases: Clinically Grounded Privacy Evaluation of Medical LMs

More like this (12)

Measuring Epistemic Resilience of LLMs Under Misleading Medical Context Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respond Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance?Your Privacy My Cloak: Backdoor Attacks on Differentially Private Federated Learning Open Medical-LLM Leaderboard Which Models Are Our Models Built On? Auditing Invisible Dependencies in Modern LLMs LLM-Guided Evolution for Medical Decision Pipelines Janus: A Benchmark for Goal-Conditioned Information Distortion in LLMs Will the Agent Recuse Itself? Measuring LLM-Agent Compliance with In-Band Access-Deny Signals LLM-augmented clinical NLP pipeline MDLM

Recent events (1)

6arXiv · cs.CL·8d ago·source ↗

Clinically grounded privacy evaluation framework reveals high memorization risk in medical LMs

Researchers introduce a tiered adversarial framework for evaluating privacy leakage in medical language models, moving beyond simple training-text recovery to realistic clinical threat models. Applied to an LM pretrained on 378k clinical notes, the framework finds that routine encounter metadata (name, DOB, provider, visit date) elicits high verbatim memorization and sensitive-diagnosis recovery (AUROC 0.91 for abortion, 0.81 for HIV). The study also finds that exact-match memorization overstates disclosure risk because 36% of memorized tokens reflect templated documentation. The work provides a practical contextual privacy evaluation methodology for medical LMs trained on longitudinal patient data.

Evaluation and Benchmarking AI Safety Research Clinically Grounded Privacy Evaluation of Medical LMs +1 more