Introducing gpt-realtime and Realtime API updates
OpenAI is releasing a new speech-to-speech model called gpt-realtime alongside expanded Realtime API capabilities. New features include MCP server support, image input, and SIP phone calling support. These updates extend the Realtime API's utility for voice-driven and multimodal agent applications.
Related guides (3)
Related events (8)
Introducing the Realtime API
OpenAI has launched the Realtime API, enabling developers to build low-latency speech-to-speech experiences directly into their applications. The API provides native audio input and output without requiring separate transcription and text-to-speech steps. This represents a significant infrastructure offering for voice-enabled AI applications, moving beyond text-based API paradigms.
GPT-Realtime-2, GPT-Translate, and new Whisper: OpenAI's new SOTA realtime voice APIs
OpenAI has released a suite of new real-time voice and audio APIs including GPT-Realtime-2, a GPT-Translate model, and an updated Whisper, all positioned as state-of-the-art for real-time voice applications. The releases appear to be part of a broader push to deploy GPT-5 capabilities across multiple product surfaces. Coverage comes from the Latent Space AI News digest, which aggregates and contextualizes the announcements.
OpenAI Updates Audio Models That Reason, Transcribe, and Translate
OpenAI introduced three new audio models in its Realtime API: GPT-Realtime-2 (speech-to-speech with five configurable reasoning effort levels), GPT-Realtime-Translate (70+ input languages), and GPT-Realtime-Whisper (transcription). GPT-Realtime-2 operates as an end-to-end audio model including reasoning, with latency ranging from 1.12 seconds at minimal effort to 2.33 seconds at high effort. Benchmark results are mixed: it leads Scale AI's Audio MultiChallenge and Artificial Analysis Conversational Dynamics but trails Step-Audio R1.1 Realtime and Grok Voice Think Fast 1.0 on speech reasoning and agentic tasks. The configurable reasoning-latency tradeoff is positioned as a key differentiator for voice agent applications.
Advancing voice intelligence with new models in the API
OpenAI is releasing new realtime voice models via its API with capabilities spanning reasoning, translation, and transcription. The announcement targets developers building voice-enabled applications and represents an expansion of OpenAI's voice intelligence offerings beyond the existing Realtime API. The models are positioned to enable more natural and intelligent voice experiences in production deployments.
Introducing ChatGPT and Whisper APIs
OpenAI announced the release of dedicated APIs for ChatGPT (gpt-3.5-turbo) and Whisper, enabling developers to integrate conversational AI and speech-to-text capabilities into their applications. The ChatGPT API offered significant cost reductions compared to existing GPT-3.5 endpoints. This marked a major step in OpenAI's platform strategy, opening programmatic access to its most widely used consumer models.
ChatGPT can now see, hear, and speak
OpenAI announced multimodal capabilities for ChatGPT, enabling the model to process images (vision), listen to voice input, and respond with synthesized speech. These features expand ChatGPT beyond text-only interaction into a multimodal assistant experience. The rollout was announced for Plus and Enterprise users first, with broader availability to follow.
OpenAI Spring Update: GPT-4o Announced, Expanded Free ChatGPT Capabilities
OpenAI announced GPT-4o, a new flagship model, alongside an expansion of capabilities available to free-tier ChatGPT users. GPT-4o represents a new omnimodal architecture capable of handling text, audio, and vision in a unified model. The announcement was made via a live demo event and marks a significant shift in OpenAI's product and model strategy.
OpenAI Introduces Next-Generation Audio Models in the API
OpenAI is releasing new audio models via its API, including an updated text-to-speech model that accepts natural-language style instructions (e.g., 'talk like a sympathetic customer service agent'). This marks the first time developers can programmatically control speaking style through prompts rather than fixed voice presets. The release targets voice agent developers seeking finer-grained customization of synthesized speech.