Designing AI agents to resist prompt injection
OpenAI published a blog post describing how ChatGPT's agent workflows are designed to resist prompt injection and social engineering attacks. The approach focuses on constraining risky actions and protecting sensitive data within agentic pipelines. This represents OpenAI's public articulation of defensive design principles for deployed AI agents.
Related guides (4)
Related events (8)
Understanding prompt injections: a frontier security challenge
OpenAI has published a blog post addressing prompt injection attacks as a key security challenge for AI systems. The post covers how these attacks work and outlines OpenAI's multi-pronged approach including research, model training improvements, and safeguard development. This signals OpenAI's formal positioning on agentic security threats as their models are increasingly deployed in tool-using and autonomous contexts.
Continuously hardening ChatGPT Atlas against prompt injection
OpenAI is applying automated red teaming trained with reinforcement learning to harden ChatGPT Atlas, its browser agent, against prompt injection attacks. The approach creates a proactive discover-and-patch loop to identify novel exploits before they can be weaponized. This work is framed as part of broader efforts to secure increasingly agentic AI systems against adversarial manipulation of external content.
Introducing Lockdown Mode and Elevated Risk Labels in ChatGPT
OpenAI is introducing two new enterprise security features in ChatGPT: Lockdown Mode, designed to help organizations defend against prompt injection attacks, and Elevated Risk labels to flag AI-driven data exfiltration attempts. These features target organizational deployments where adversarial manipulation of AI systems poses operational security risks. The announcement signals growing attention to agentic and enterprise threat models within ChatGPT's product surface.
Keeping your data safe when an AI agent clicks a link
OpenAI published a blog post describing safeguards built into its AI agent systems to prevent URL-based data exfiltration and prompt injection attacks when agents follow links. The post outlines how OpenAI protects user data during agentic browsing or link-following actions. This addresses a known attack surface in autonomous agent deployments where malicious links could be used to leak context or hijack agent behavior.
Introducing ChatGPT Agent
OpenAI has launched ChatGPT agent, a new capability that combines reasoning with tool use to autonomously complete multi-step tasks such as research, bookings, and presentation creation. The agent operates under user guidance, integrating thinking and acting in a unified workflow. This represents OpenAI's move to bring agentic capabilities directly into the ChatGPT product for general consumers.
ChatGPT Agent System Card
OpenAI has published a system card for its ChatGPT agent, an agentic model that integrates research, browser automation, and code execution tools into a unified system. The release is accompanied by safety documentation under OpenAI's Preparedness Framework. The system card details the safeguards and evaluations applied to the agent prior to deployment. This represents OpenAI's formal safety disclosure for a production agentic product.
How should AI systems behave, and who should decide?
OpenAI published a policy post clarifying how ChatGPT's behavior is shaped and governed, outlining plans to allow greater user customization of model behavior. The post also describes intentions to solicit broader public input into decision-making around AI system behavior. This represents an early public articulation of OpenAI's approach to behavioral governance and value alignment in deployed systems.
Introducing ChatGPT
OpenAI announced ChatGPT, a conversational model trained to engage in dialogue, answer follow-up questions, acknowledge errors, challenge incorrect premises, and decline inappropriate requests. The model's dialogue format represented a significant step in making large language models accessible and interactive for general users. This November 2022 launch marked a pivotal moment in public AI adoption.