Entity · paper

Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respond

paperactivesecurity-and-privacy-prompts-in-the-wild-what-users-ask-llms-and-how-llms-respond-9f470643·1 events·first seen Jun 17, 2026

Aliases: Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respond

Co-occurring entities

WildChat Llama GPT-5.5

More like this (12)

Online Safety Monitoring for LLMs Prompting Complexity: Shortest Prompts for Texts and Behaviors in LLMs Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues Can LLMs Reliably Self-Report Adversarial Prefills, and How?Clinically Grounded Privacy Evaluation of Medical LMs What Do Safety-Aligned LLMs Learn From Mixed Compliance Demonstrations?Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research LLM Detection as an Intervention: Downstream Impact under Strategic User Behavior The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs Measuring Epistemic Resilience of LLMs Under Misleading Medical Context frontier LLMs RAS: Measuring LLM Safety Through Refusal Alignment

Recent events (1)

5arXiv · cs.CL·Jun 17, 2026·source ↗

Study of security and privacy prompts in the wild reveals LLM response quality gaps and inconsistency

Researchers analyzed 14,727 security and privacy (S&P) prompts drawn from WildChat's 3.2M real user-LLM conversations, categorizing them into nine topic areas and evaluating response quality across 270 advice-seeking prompts. Commercial models substantially outperformed open-weight models (GPT achieving 98% 'good enough' responses vs. Llama 4 at 47%), but even high-performing commercial models showed inconsistent responses across repeated runs of the same prompt. The study is the first to analyze real user S&P queries to LLMs rather than expert-authored test sets, surfacing both a capability gap and a reliability concern.

Evaluation and Benchmarking AI Safety Research Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respond WildChat Llama +1 more