openai · OpenAI Platform Docs
Realtime conversations | OpenAI API
Teaches how to implement low-latency, multimodal voice and text interactions using the Realtime API, covering session management and event-driven communication.
Derived skill
Files assembled from official documentation
Viewing SKILL.md
Realtime conversations | OpenAI API
Teaches how to implement low-latency, multimodal voice and text interactions using the Realtime API, covering session management and event-driven communication.
When To Use
Use when implementing low-latency, bidirectional voice or text streaming interfaces that require real-time model responses.
Reference Files
| File | Contains | Use For |
|---|---|---|
SKILL.md | Entry point: scope, routing table, and workflow. | Start here. |
docs/realtime-conversations-openai-api-workflow-guide.md | A guide detailing the lifecycle, session events, and input/output handling for speech-to-speech conversations using the OpenAI Realtime API. | Questions about a guide detailing the lifecycle, session events, and input/output handling for speech-to-speech conversations using t... |
examples/realtime-conversations-openai-api-openai-realtime-api-session-update-eve.text | A JSON object representing a session.update event configuration for the OpenAI Realtime API, specifying audio output modalities and PCM audio input formats. | Exact payloads, commands, or snippets shown in A JSON object representing a session.update event configuration for the OpenAI Realtime API, specifying audio output... |
examples/realtime-conversations-openai-api-openai-realtime-api-session-update-eve-2.text | A JSON object representing a session.update event configuration for the OpenAI Realtime API, specifying audio output modalities and PCM audio input settings. | Exact payloads, commands, or snippets shown in A JSON object representing a session.update event configuration for the OpenAI Realtime API, specifying audio output... |
examples/realtime-conversations-openai-api-conversation-item-create-event.text | A JSON event object demonstrating the conversation.item.create structure for sending a user message via a WebRTC data channel. | Exact payloads, commands, or snippets shown in A JSON event object demonstrating the conversation.item.create structure for sending a user message via a WebRTC data... |
examples/realtime-conversations-openai-api-openai-realtime-api-conversation-item-.text | A JSON event structure demonstrating how to create a new conversation item with user text input for the OpenAI Realtime API. | Exact payloads, commands, or snippets shown in A JSON event structure demonstrating how to create a new conversation item with user text input for the OpenAI Realti... |
examples/realtime-conversations-openai-api-response-create-event-json.text | A JSON object representing a response.create event used to trigger a model response in a realtime conversation. | Exact payloads, commands, or snippets shown in A JSON object representing a response.create event used to trigger a model response in a realtime conversation. |
examples/realtime-conversations-openai-api-response-create-event.text | A JSON event object demonstrating the response.create event type for initiating a response in a realtime conversation. | Exact payloads, commands, or snippets shown in A JSON event object demonstrating the response.create event type for initiating a response in a realtime conversation. |
examples/realtime-conversations-openai-api-realtime-conversations-webrtc-event-ha.text | A JavaScript function that parses and logs server events from a WebRTC data channel during a realtime conversation. | Exact payloads, commands, or snippets shown in A JavaScript function that parses and logs server events from a WebRTC data channel during a realtime conversation. |
examples/realtime-conversations-openai-api-python-onmessage-handler.text | A Python function implementation for handling WebSocket server events and printing response output in a realtime conversation. | Exact payloads, commands, or snippets shown in A Python function implementation for handling WebSocket server events and printing response output in a realtime conv... |
examples/realtime-conversations-openai-api-webrtc-peer-connection-javascript.text | A JavaScript implementation demonstrating how to establish an RTCPeerConnection to handle remote audio playback and local microphone input for Realtime API conversations. | Exact payloads, commands, or snippets shown in A JavaScript implementation demonstrating how to establish an RTCPeerConnection to handle remote audio playback and l... |
examples/realtime-conversations-openai-api-openai-realtime-api-float32-to-pcm16-c.text | A JavaScript function that converts a Float32Array of audio data into a PCM16 ArrayBuffer. | Exact payloads, commands, or snippets shown in A JavaScript function that converts a Float32Array of audio data into a PCM16 ArrayBuffer. |
examples/realtime-conversations-openai-api-openai-realtime-api-python-websocket-c.text | A Python script demonstrating how to establish a WebSocket connection to the OpenAI Realtime API and process audio data. | Exact payloads, commands, or snippets shown in A Python script demonstrating how to establish a WebSocket connection to the OpenAI Realtime API and process audio data. |
examples/realtime-conversations-openai-api-openai-realtime-api-conversation-item--2.text | A JSON event object demonstrating how to create a conversation item with input audio for the OpenAI Realtime API. | Exact payloads, commands, or snippets shown in A JSON event object demonstrating how to create a conversation item with input audio for the OpenAI Realtime API. |
examples/realtime-conversations-openai-api-openai-realtime-api-conversation-item--3.text | A JSON-formatted WebSocket event demonstrating how to create a conversation item with input audio content using the OpenAI Realtime API. | Exact payloads, commands, or snippets shown in A JSON-formatted WebSocket event demonstrating how to create a conversation item with input audio content using the O... |
examples/realtime-conversations-openai-api-openai-realtime-api-websocket-event-ha.text | A JavaScript function demonstrating how to parse and handle incoming WebSocket server events, specifically targeting audio delta chunks, for the OpenAI Realtime API. | Exact payloads, commands, or snippets shown in A JavaScript function demonstrating how to parse and handle incoming WebSocket server events, specifically targeting... |
examples/realtime-conversations-openai-api-openai-realtime-api-python-onmessage-a.text | A Python function demonstrating how to process and access Base64-encoded audio delta chunks from a Realtime API WebSocket message event. | Exact payloads, commands, or snippets shown in A Python function demonstrating how to process and access Base64-encoded audio delta chunks from a Realtime API WebSo... |
examples/realtime-conversations-openai-api-openai-realtime-api-conversation-item--4.text | A JSON event object demonstrating how to create a conversation item with an input image using the OpenAI Realtime API. | Exact payloads, commands, or snippets shown in A JSON event object demonstrating how to create a conversation item with an input image using the OpenAI Realtime API. |
examples/realtime-conversations-openai-api-openai-realtime-api-response-create-ev.text | A text example demonstrating the structure of a response.create event for managing out-of-band responses in the OpenAI Realtime API. | Exact payloads, commands, or snippets shown in A text example demonstrating the structure of a response.create event for managing out-of-band responses in the OpenA... |
examples/realtime-conversations-openai-api-openai-realtime-api-response-create-ev-2.text | A text example demonstrating the structure of a response.create event within the OpenAI Realtime API. | Exact payloads, commands, or snippets shown in A text example demonstrating the structure of a response.create event within the OpenAI Realtime API. |
examples/realtime-conversations-openai-api-openai-realtime-api-webrtc-event-handl.text | A JavaScript function that parses WebRTC server events to handle response metadata and output for realtime conversations. | Exact payloads, commands, or snippets shown in A JavaScript function that parses WebRTC server events to handle response metadata and output for realtime conversati... |
examples/realtime-conversations-openai-api-openai-realtime-api-python-message-han.text | A Python code snippet demonstrating how to handle WebSocket server events and metadata classification within an OpenAI Realtime API session. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to handle WebSocket server events and metadata classification within an OpenA... |
examples/realtime-conversations-openai-api-openai-realtime-api-response-create-ev-3.text | A JSON-formatted event object demonstrating how to use the response.create type within the OpenAI Realtime API. | Exact payloads, commands, or snippets shown in A JSON-formatted event object demonstrating how to use the response.create type within the OpenAI Realtime API. |
examples/realtime-conversations-openai-api-openai-realtime-api-response-create-ev-4.text | A JSON object representing a response.create event used to trigger a model response in a realtime conversation. | Exact payloads, commands, or snippets shown in A JSON object representing a response.create event used to trigger a model response in a realtime conversation. |
examples/realtime-conversations-openai-api-openai-realtime-api-response-create-ev-5.text | A JavaScript code snippet demonstrating how to construct a response.create event with custom instructions for the OpenAI Realtime API. | Exact payloads, commands, or snippets shown in A JavaScript code snippet demonstrating how to construct a response.create event with custom instructions for the Ope... |
examples/realtime-conversations-openai-api-openai-realtime-api-python-response-cr.text | A Python code snippet demonstrating how to send a response.create event with custom instructions to the OpenAI Realtime API via a WebSocket. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to send a response.create event with custom instructions to the OpenAI Realti... |
examples/realtime-conversations-openai-api-openai-realtime-api-session-update-fun.text | A text representation of a session.update event configuring a function tool named generatehoroscope for the Realtime API. | Exact payloads, commands, or snippets shown in A text representation of a session.update event configuring a function tool named generatehoroscope for the Realtime... |
examples/realtime-conversations-openai-api-openai-realtime-api-conversation-item--5.text | A JSON-formatted event demonstrating the conversation.item.create operation for adding a user message to a realtime session. | Exact payloads, commands, or snippets shown in A JSON-formatted event demonstrating the conversation.item.create operation for adding a user message to a realtime s... |
examples/realtime-conversations-openai-api-openai-realtime-api-response-create-ev-6.text | A text representation of a response.create event used to trigger a model response in a realtime conversation. | Exact payloads, commands, or snippets shown in A text representation of a response.create event used to trigger a model response in a realtime conversation. |
examples/realtime-conversations-openai-api-openai-realtime-api-response-done-even.text | A text representation of a response.done event object from the OpenAI Realtime API showing completed response status and output items. | Exact payloads, commands, or snippets shown in A text representation of a response.done event object from the OpenAI Realtime API showing completed response status... |
examples/realtime-conversations-openai-api-openai-realtime-api-conversation-item--6.text | A JSON-formatted text snippet demonstrating a conversation.item.create event with a functioncalloutput type for the OpenAI Realtime API. | Exact payloads, commands, or snippets shown in A JSON-formatted text snippet demonstrating a conversation.item.create event with a functioncalloutput type for the O... |
examples/realtime-conversations-openai-api-event-structure.text | A text representation of a JSON event object used for sending data through a data channel in a realtime conversation. | Exact payloads, commands, or snippets shown in A text representation of a JSON event object used for sending data through a data channel in a realtime conversation. |
examples/realtime-conversations-openai-api-openai-realtime-api-invalid-requesterr.text | An example of an invalidrequesterror event containing an invalid value error within a realtime API conversation. | Exact payloads, commands, or snippets shown in An example of an invalidrequesterror event containing an invalid value error within a realtime API conversation. |
examples/realtime-conversations-openai-api-openai-realtime-api-conversation-item--7.text | A JSON event structure demonstrating how to truncate a conversation item and its associated audio content using the conversation.item.truncate type. | Exact payloads, commands, or snippets shown in A JSON event structure demonstrating how to truncate a conversation item and its associated audio content using the c... |
What This Skill Covers
- Once you have connected to the Realtime API through either WebRTC or WebSocket, you can call a Realtime model (such as gpt-realtime-2) to have speech-to-spee...
- Main sections:
Realtime speech-to-speech sessions,Session lifecycle events,Text inputs and outputs,Audio inputs and outputs,Voice options.
Workflow
- Open the most relevant file under
docs/for the exact documented workflow and wording. - Open
schemas/files for exact structured contracts. - Open
examples/files for concrete requests, commands, snippets, and manifests. - Do not add behavior or configuration that is not present in the attached source files.
Canonical source: https://developers.openai.com/api/docs/guides/realtime-conversations