Entity · paper

AISPA: User-Centric System Prompt Auditing for Large Language Model Applications

paperactiveprovisionalaispa-user-centric-system-prompt-auditing-for-large-language-model-applications-24f75e98·1 events·first seen 4h ago

Aliases: AISPA: User-Centric System Prompt Auditing for Large Language Model Applications

Co-occurring entities

AISPA

More like this (12)

Automated reproducibility assessments in the social and behavioral sciences using large language models Auditing Protocol-Level Shortcuts in Large Audio Language Model Judges for Speech Evaluation The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models Selective Disclosure Watermarking for Large Language Models Understanding Large Language Models Sound Probabilistic Safety Bounds for Large Language Models Civil Court Simulation with Large Language Models Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research On the Limits of Prompt-Conditioned Language Models as General-Purpose Learners Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application Situated Interaction Auditing

Recent events (1)

6arXiv · cs.CL·4h ago·source ↗

AISPA framework audits system prompts across 88 commercial AI products, finds widespread problematic instructions

Researchers introduce AISPA (Artificial Intelligence System Prompt Assurance), a framework for auditing system prompts in commercial AI applications across eight user-centric dimensions. An audit of 3,249 instructions from 88 commercial products finds that while 98.9% contain at least one protective instruction, roughly 40% also contain at least one instruction that works against user interests. System prompt design varies dramatically across organizations, and only 24% of products cover all eight AISPA dimensions. The findings argue for greater transparency, standardization, and independent oversight of system prompts.

AI Safety Research Enterprise Deployment Patterns AISPA: User-Centric System Prompt Auditing for Large Language Model Applications AISPA +1 more