google · Google AI Docs
Gemini API Live API capabilities guide
An overview of the multimodal and functional features supported by the Gemini Live API, including tool use, session management, and various input/output modalities like audio, video, and text.
Derived skill
Files assembled from official documentation
Viewing SKILL.md
Gemini API Live API capabilities guide
An overview of the multimodal and functional features supported by the Gemini Live API, including tool use, session management, and various input/output modalities like audio, video, and text.
When To Use
Use when determining which multimodal inputs, outputs, or tool-calling features are available to implement a real-time interactive agent using the Live API.
Reference Files
| File | Contains | Use For |
|---|---|---|
SKILL.md | Entry point: scope, routing table, and workflow. | Start here. |
docs/gemini-api-live-capabilities-guide-workflow-guide.md | A comprehensive guide detailing the capabilities, model comparisons, and connection configurations for the Gemini API Live API. | Questions about a comprehensive guide detailing the capabilities, model comparisons, and connection configurations for the Gemini API... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-python-multimodal-aud.text | A Python script demonstrating how to establish an asynchronous connection to the Gemini Live API using the Google GenAI SDK to receive audio responses. | Exact payloads, commands, or snippets shown in A Python script demonstrating how to establish an asynchronous connection to the Gemini Live API using the Google Gen... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-modality-nodejs.text | A Node.js code example demonstrating how to configure the Gemini API Live session to use audio as a response modality. | Exact payloads, commands, or snippets shown in A Node.js code example demonstrating how to configure the Gemini API Live session to use audio as a response modality. |
examples/gemini-api-live-capabilities-guide-gemini-api-live-realtime-audio-input-.text | A Python code snippet demonstrating how to send raw PCM audio bytes to the Gemini API Live session using the send_realtime_input method. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to send raw PCM audio bytes to the Gemini API Live session using the sendreal... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-realtime-audio-input--2.text | A Node.js code snippet demonstrating how to send raw PCM audio chunks to the Gemini API Live session using base64 encoding. | Exact payloads, commands, or snippets shown in A Node.js code snippet demonstrating how to send raw PCM audio chunks to the Gemini API Live session using base64 enc... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-session-receive-audio.text | A Python code snippet demonstrating how to asynchronously iterate through a Live API session to receive and process model-generated audio data. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to asynchronously iterate through a Live API session to receive and process m... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-onmessage-audio-data-.text | A JavaScript code snippet demonstrating how to extract and process base64 encoded audio data from the serverContent object within an onmessage callback. | Exact payloads, commands, or snippets shown in A JavaScript code snippet demonstrating how to extract and process base64 encoded audio data from the serverContent o... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-realtime-input.text | A text-based code example demonstrating how to use the send_realtime_input method to send text to a Gemini API Live session. | Exact payloads, commands, or snippets shown in A text-based code example demonstrating how to use the sendrealtimeinput method to send text to a Gemini API Live ses... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-realtime-input-sessio.text | A text example demonstrating the use of the session.sendRealtimeInput method to send text input to the Gemini API Live session. | Exact payloads, commands, or snippets shown in A text example demonstrating the use of the session.sendRealtimeInput method to send text input to the Gemini API Liv... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-realtime-video-input-.text | A Python code snippet demonstrating how to send JPEG-encoded image bytes as realtime video input to the Gemini API Live session. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to send JPEG-encoded image bytes as realtime video input to the Gemini API Li... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-realtime-video-input--2.text | A code snippet demonstrating how to send base64-encoded JPEG image data as realtime video input using the Gemini API Live session. | Exact payloads, commands, or snippets shown in A code snippet demonstrating how to send base64-encoded JPEG image data as realtime video input using the Gemini API... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-multimodal-session-in.text | A text-based example demonstrating how to send client content turns to a Gemini API Live session using asynchronous method calls. | Exact payloads, commands, or snippets shown in A text-based example demonstrating how to send client content turns to a Gemini API Live session using asynchronous m... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-send-client-javascrip.text | A JavaScript code example demonstrating how to use the session.sendClientContent method to send multi-turn conversation history to the Gemini Live API. | Exact payloads, commands, or snippets shown in A JavaScript code example demonstrating how to use the session.sendClientContent method to send multi-turn conversati... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-connection-pyth.text | A Python code example demonstrating how to establish an asynchronous connection to the Gemini Live API using the Google GenAI SDK to send client content and receive audio responses. | Exact payloads, commands, or snippets shown in A Python code example demonstrating how to establish an asynchronous connection to the Gemini Live API using the Goog... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-nodejs-audio-modality.text | A Node.js code example demonstrating how to configure the Gemini API Live API to use audio response modalities. | Exact payloads, commands, or snippets shown in A Node.js code example demonstrating how to configure the Gemini API Live API to use audio response modalities. |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-connection-pyth-2.text | A Python script demonstrating how to establish a real-time bidirectional audio connection using the Gemini API Live client. | Exact payloads, commands, or snippets shown in A Python script demonstrating how to establish a real-time bidirectional audio connection using the Gemini API Live c... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-modality-nodejs-2.text | A Node.js code example demonstrating how to configure the Gemini API Live API to use audio response modalities and input audio transcription. | Exact payloads, commands, or snippets shown in A Node.js code example demonstrating how to configure the Gemini API Live API to use audio response modalities and in... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-response-config.text | A configuration object defining audio response modalities and prebuilt voice settings for the Gemini API Live API. | Exact payloads, commands, or snippets shown in A configuration object defining audio response modalities and prebuilt voice settings for the Gemini API Live API. |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-modality-config.text | A configuration object demonstrating how to set response modalities to audio and specify a prebuilt voice name for the Gemini API Live API. | Exact payloads, commands, or snippets shown in A configuration object demonstrating how to set response modalities to audio and specify a prebuilt voice name for th... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-connect-python-audio-.text | A Python code example demonstrating how to initialize a Gemini API Live session with audio response modalities and thinking configuration. | Exact payloads, commands, or snippets shown in A Python code example demonstrating how to initialize a Gemini API Live session with audio response modalities and th... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-modality-nodejs-3.text | A Node.js code example demonstrating how to configure the Gemini API Live API to use audio response modalities. | Exact payloads, commands, or snippets shown in A Node.js code example demonstrating how to configure the Gemini API Live API to use audio response modalities. |
examples/gemini-api-live-capabilities-guide-gemini-api-live-config-audio-thinking.text | A Python configuration snippet for the Gemini Live API specifying audio response modalities and low-level thinking parameters. | Exact payloads, commands, or snippets shown in A Python configuration snippet for the Gemini Live API specifying audio response modalities and low-level thinking pa... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-capabilities-config.text | A text configuration example demonstrating response modalities and thinking configuration settings for the Gemini API Live API. | Exact payloads, commands, or snippets shown in A text configuration example demonstrating response modalities and thinking configuration settings for the Gemini API... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-config-python-client-.text | A Python code snippet demonstrating how to initialize a Gemini API client with Live API configuration settings including audio response modalities and affective dialogue. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to initialize a Gemini API client with Live API configuration settings includ... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-api-config-options-ja.text | A JavaScript code snippet demonstrating configuration options for the Gemini API Live API, including response modalities and affective dialog settings. | Exact payloads, commands, or snippets shown in A JavaScript code snippet demonstrating configuration options for the Gemini API Live API, including response modalit... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-config-python-client--2.text | A Python code snippet demonstrating how to initialize a Gemini API client with Live API configuration and audio response modalities. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to initialize a Gemini API client with Live API configuration and audio respo... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-modality-config-2.text | A JavaScript code snippet demonstrating how to configure response modalities and proactive audio settings for the Gemini API Live API. | Exact payloads, commands, or snippets shown in A JavaScript code snippet demonstrating how to configure response modalities and proactive audio settings for the Gem... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-session-receive-inter.text | A Python code snippet demonstrating how to handle server-side interruptions during a Gemini API Live session using an asynchronous loop. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to handle server-side interruptions during a Gemini API Live session using an... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-api-interruption-hand.text | A JavaScript code snippet demonstrating how to detect and handle server-side interruptions during real-time audio playback in the Gemini Live API. | Exact payloads, commands, or snippets shown in A JavaScript code snippet demonstrating how to detect and handle server-side interruptions during real-time audio pla... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-pcm-sample.text | A Python script demonstrating how to use a PCM audio file with the Gemini API Live API. | Exact payloads, commands, or snippets shown in A Python script demonstrating how to use a PCM audio file with the Gemini API Live API. |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-modality-nodejs-4.text | A Node.js code example demonstrating how to configure the Gemini API Live API to use audio as a response modality with a sample PCM audio file. | Exact payloads, commands, or snippets shown in A Node.js code example demonstrating how to configure the Gemini API Live API to use audio as a response modality wit... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-python-config-modalit.text | A Python code snippet demonstrating how to configure response modalities and automatic activity detection for the Gemini API Live API. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to configure response modalities and automatic activity detection for the Gem... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-config-nodejs.text | A Node.js code example demonstrating how to configure response modalities and automatic activity detection for the Gemini API Live API. | Exact payloads, commands, or snippets shown in A Node.js code example demonstrating how to configure response modalities and automatic activity detection for the Ge... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-modalities-pyth.text | A Python code snippet demonstrating how to configure the Gemini API Live session with audio response modalities and automatic activity detection settings. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to configure the Gemini API Live session with audio response modalities and a... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-modality-config-3.text | A configuration object demonstrating how to enable audio response modalities and manual realtime input control for the Gemini API Live API. | Exact payloads, commands, or snippets shown in A configuration object demonstrating how to enable audio response modalities and manual realtime input control for th... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-session-usage-metadat.text | A Python code snippet demonstrating how to iterate through a Live API session to receive and parse UsageMetadata for token counts. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to iterate through a Live API session to receive and parse UsageMetadata for... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-api-usage-metadata-no.text | A Node.js code example demonstrating how to access and log usage metadata and token breakdown from Gemini API Live API turns. | Exact payloads, commands, or snippets shown in A Node.js code example demonstrating how to access and log usage metadata and token breakdown from Gemini API Live AP... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-audio-modality-config-4.text | A Python code snippet demonstrating how to configure response modalities and media resolution for the Gemini API Live API. | Exact payloads, commands, or snippets shown in A Python code snippet demonstrating how to configure response modalities and media resolution for the Gemini API Live... |
examples/gemini-api-live-capabilities-guide-gemini-api-live-capabilities-nodejs-c.text | A Node.js code snippet demonstrating configuration options for the Gemini API Live API, including response modalities and media resolution settings. | Exact payloads, commands, or snippets shown in A Node.js code snippet demonstrating configuration options for the Gemini API Live API, including response modalities... |
What This Skill Covers
- Preview: The Live API is in preview.
- Main sections:
Before you begin,Model comparison,Establishing a connection,Python,JavaScript.
Workflow
- Open the most relevant file under
docs/for the exact documented workflow and wording. - Open
schemas/files for exact structured contracts. - Open
examples/files for concrete requests, commands, snippets, and manifests. - Do not add behavior or configuration that is not present in the attached source files.
Canonical source: https://ai.google.dev/gemini-api/docs/live-api/capabilities