> For the complete documentation index, see [llms.txt](https://docs.concurrence.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.concurrence.com/developer-guide/guides/app-voice-session.md).

# 8. Run a Voice Session in Your App

Start a hold-to-talk voice session from your own app: your backend gets a one-use ticket, the app streams PCM16 audio over a WebSocket, and the agent answers on the same socket.

A voice session is one live voice conversation between your app and a voice service. App voice sessions are a beta feature, with limits on how many can run at once per workspace and per person. The app streams the user's microphone to a WebSocket, the agent's voice comes back on the same socket, and transcripts and timing frames arrive as small JSON messages. The agent is the service's own agent: it can use the service's tools, and the session is recorded and billed like a call.

## Before You Start

You need:

* A voice service on `gpt_realtime`, `eleven_realtime`, `gpt_live`, or `grok_voice`. Any other service is refused with `422`. See [Choosing a Realtime Voice Family](https://docs.concurrence.com/developer-guide/platform-api/workspaces/voice-configuration.md#choosing-a-realtime-voice-family) for channels and graph requirements, and [Session Provider](https://docs.concurrence.com/developer-guide/platform-api/workspaces/services.md#session-provider) for save refusals.
* A workspace API key whose role is member or above. Keep it on your backend. Do not ship it in the app.
* The agent engine host for your region, `https://agent-engine.api-us.concurrence.com` in US production. See [Regions & Endpoints](https://docs.concurrence.com/developer-guide/getting-started/regions-and-endpoints.md#service-specific-hosts).

These operations are not published in the Platform OpenAPI document, so this page describes their contract directly.

## How a Session Runs

1. Your app asks your backend for a session.
2. Your backend calls the start route with the API key and receives a `ws_url` and a one-use `ticket`.
3. Your backend hands both to the app.
4. The app opens the WebSocket, waits for `ready`, then streams audio every 20 ms until it hangs up.

On `gpt_live`, the opening AI and recording notice is prerecorded, as are the emergency scripts and a small set of fixed lines. The caller cannot interrupt these prerecorded lines, and speech during playback is not processed. The timeout line is English only. The session uses Concurrence reasoning for multi-state Context Graph navigation on delegated requests and is billed as voice minutes. See [GPT-Live](https://docs.concurrence.com/developer-guide/platform-api/workspaces/voice-configuration.md#gpt-live) for configuration and channel scope.

The agent speaks first. It plays the spoken notice that the call is with an AI and is recorded, then its greeting if the service has one. The notice plays at the start of a session when it can be rendered. On `eleven_realtime`, it is spoken in the selected ElevenLabs voice. On `grok_voice`, it is prerecorded, as are the emergency scripts and a small set of fixed lines, and the rest of the session uses the selected Grok voice. For a service that needs the notice spoken word for word, prefer `gpt_realtime` and verify the opening with representative calls.

## Start a Session

Call `POST /agent/voice-session/start` from your backend:

```http
POST /agent/voice-session/start
Authorization: Bearer <workspace API key>
Content-Type: application/json
```

```json
{
  "service_id": "00000000-0000-4000-8000-000000000002",
  "subject": {"external_subject_key": "user-1", "caller_id": "+12025550100"},
  "audio": {"input": {"encoding": "pcm16", "sample_rate": 24000}},
  "turn_mode": "hold_to_talk"
}
```

* `service_id` is the voice service to run.
* `subject.external_subject_key` is 1 to 128 characters: your stable id for the end user. It caps concurrent sessions per person. The platform does not verify it.
* `subject.caller_id` is optional, an E.164 number the session treats as the caller's phone number, as on a phone call. It is not verified and does not prove who is speaking.
* `audio` and `turn_mode` can be left out. They accept only the values shown.
* Any other field is refused with `422`. A request cannot change the agent's prompt or configuration.

The response:

```json
{
  "ws_url": "wss://<voice host>/agent/voice-session",
  "subprotocol": "auth",
  "ticket": "vst_...",
  "call_sid": "vs-0123456789abcdef0123456789abcdef",
  "expires_in": 60,
  "audio": {
    "input": {"encoding": "pcm16", "sample_rate": 24000, "channels": 1},
    "output": {"encoding": "pcm16", "sample_rate": 24000, "channels": 1}
  },
  "turn_mode": "hold_to_talk"
}
```

The ticket works for one connection and for 60 seconds. `call_sid` identifies the session: it is the `session_id` in the `ready` frame and the id the conversation is recorded under.

| Status | `detail.code`                                                              | Meaning                                                                                                                                                                                        | What to do                                                                            |
| ------ | -------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| 401    | none                                                                       | The API key is missing or wrong                                                                                                                                                                | Fix the key                                                                           |
| 403    | none                                                                       | The credential may not use this route                                                                                                                                                          | Use a workspace API key                                                               |
| 403    | `voice_session_permission_required`                                        | The key's role is below member                                                                                                                                                                 | Use a key with member role or above                                                   |
| 403    | `eleven_realtime_not_enabled`                                              | The service is on `eleven_realtime` and the workspace's `data_residency` is `regional`                                                                                                         | Use a service on another realtime voice family                                        |
| 403    | `gpt_live_region_unsupported`                                              | The service uses `gpt_live` outside the US                                                                                                                                                     | Use a US workspace, or choose another family                                          |
| 403    | `gpt_live_unavailable`                                                     | The service uses `gpt_live` and the workspace has regional data residency                                                                                                                      | Choose a family that meets the workspace's residency policy                           |
| 403    | `grok_voice_not_enabled`                                                   | The service is on `grok_voice` and the workspace's `data_residency` is `regional`                                                                                                              | Choose a family that meets the workspace's residency policy                           |
| 404    | `service_not_found`                                                        | No active service with this id in the workspace, or the service has no agent or Context Graph version                                                                                          | Check `service_id` and the service's configuration                                    |
| 422    | `unsupported_service_runtime`                                              | The service is not on a supported realtime voice family, or an `eleven_realtime` service has no voice configured                                                                               | Select a supported family and configure its voice on the service                      |
| 422    | `direct_program_unsupported`                                               | The service's agent version runs a [direct program](https://docs.concurrence.com/developer-guide/platform-api/workspaces/direct-programs.md#grok-voice) and the service is not on `grok_voice` | Put the service on `grok_voice`, or select an agent version that runs a Context Graph |
| 429    | `workspace_concurrency_limit`                                              | The workspace has too many live sessions                                                                                                                                                       | Wait and retry                                                                        |
| 429    | `subject_concurrency_limit`                                                | This person has too many live sessions                                                                                                                                                         | End one or wait                                                                       |
| 429    | `start_rate_limit`                                                         | The key started too many sessions in the last minute                                                                                                                                           | Wait for the `Retry-After` seconds                                                    |
| 503    | `no_voice_capacity`                                                        | No voice capacity was free                                                                                                                                                                     | Retry once after the `Retry-After` seconds                                            |
| 503    | `config_unavailable`, `slot_store_unavailable`, `ticket_store_unavailable` | A dependency was not available                                                                                                                                                                 | Retry shortly                                                                         |
| 503    | `residency_unavailable`                                                    | The workspace's data residency could not be checked for an `eleven_realtime`, `gpt_live`, or `grok_voice` service                                                                              | Retry after the `Retry-After` seconds                                                 |

For `gpt_live_region_unsupported`, `detail.message` is `GPT-Live is US only for now.` For `gpt_live_unavailable`, it is `GPT-Live requires a US workspace without regional data residency`. For `grok_voice_not_enabled`, it is `Grok Voice is not available to a workspace with regional data residency`. For `residency_unavailable`, it is `Voice session availability could not be checked; retry`.

The `401` and the `403` without a code have a plain string as `detail`. A `422` for a body that does not fit the request has a list of problems as `detail`. Every other error has `detail.code` and `detail.message`.

## Open the Socket

Offer the ticket only in the `Sec-WebSocket-Protocol` header, as the two subprotocols `auth` and the ticket, in that order:

```javascript
const socket = new WebSocket(start.ws_url, ["auth", start.ticket]);
socket.binaryType = "arraybuffer";
```

* Keep the ticket out of the URL, a query string, and cookies, and do not log it.
* The server answers with the `auth` subprotocol. Treat a socket with no selected subprotocol as failed.
* Open the socket only when you are ready to stream. The ticket is spent when the server reads it, so a reconnect is a new start call, and the opening notice plays again.

## Stream Audio

Both directions are binary frames of PCM16, little-endian, mono, 24000 Hz.

* **Client to server.** Send 20 ms frames of 480 samples (960 bytes). A frame must hold whole samples and be at most 9600 bytes. Anything else closes the socket with `4008`. Do not send empty frames.
* **Server to client.** The agent's voice arrives in frames of varying size, normally a whole number of samples. If you re-chunk, keep sample alignment.
* **Echo cancellation.** Turn it on for the microphone, or the agent hears itself while the user holds the button.

### Hold to talk

The session is hold to talk. The user holds a button while speaking and releases it to finish. Your app sends the microphone's samples while the button is held and zero samples otherwise, one frame every 20 ms at real-time pace from `ready` until the session ends.

Keep streaming the zero frames after a release. The agent decides that the user has finished from the silence that follows the speech, so a socket that goes quiet does not end the turn. Because the decision rests on silence, a pause of a few hundred milliseconds in the middle of a press can end the turn early.

## Frames

### From the app

Each is a JSON text frame with a `type`. Other text frames are ignored.

| Frame                     | When to send it                                                                                       |
| ------------------------- | ----------------------------------------------------------------------------------------------------- |
| `{"type": "input.start"}` | The button is pressed. Send it before the first audio frame of the press. It does not clear playback. |
| `{"type": "input.end"}`   | The button is released. Keep streaming zero frames.                                                   |
| `{"type": "stop"}`        | Hang up. The server answers `ended` with reason `client_stop` and closes with `1000`.                 |

### From the server

A client ignores any frame whose `type` it does not know.

| Frame           | Meaning                                                                                                                                                                                                                                                             |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ready`         | The first message sent. Carries `protocol` (`amigo-voice-session.v1`), `session_id`, the audio formats, `turn_mode`, and `limits`. You can start streaming.                                                                                                         |
| `call_progress` | `{"phase": "live"}`: the agent session is set up and the opening starts. Keep the hold button disabled until it arrives.                                                                                                                                            |
| `transcript`    | A complete utterance, not a partial one. `text` is the utterance and `role` is `user` or `agent`. An agent line carries `interrupted`, true when the reply had already been cut off when its transcript was sent. A cut that comes later arrives as `interruption`. |
| `interruption`  | The agent's speech was cut off, usually because the user barged in. Clear the playback queue immediately.                                                                                                                                                           |
| `latency`       | The server's measure from the end of the user's turn to the first agent audio it forwarded, as `e2e_ttfb_ms`. `is_greeting` is true for the opening, which is measured from when the agent session starts to open. It leaves out your network and playback buffer.  |
| `error`         | A fatal condition with a `code`: `bad_audio_frame`, `idle_timeout`, `service_not_ready`, `unsupported_service_runtime`, or `internal_error`. An `ended` frame and a close follow.                                                                                   |
| `ended`         | The last frame before the close. `reason` is `client_stop`, `idle_timeout`, `max_duration`, or `server_ended`. Treat a reason you do not know as `server_ended`.                                                                                                    |

Transcripts contain the conversation. Show them only to the person in the session and treat them as sensitive. Nothing about the agent's tools or configuration is sent to the app.

### Close codes

| Code   | Meaning                                                                                                    |
| ------ | ---------------------------------------------------------------------------------------------------------- |
| `1000` | Normal end, after `ended`.                                                                                 |
| `1011` | Server error.                                                                                              |
| `1013` | The host could not take the session: no capacity, or a dependency was unavailable. Start a new session.    |
| `4001` | No ticket was offered.                                                                                     |
| `4003` | The ticket is malformed, unknown, expired, or already used. Start a new session.                           |
| `4008` | Audio was refused: a frame of odd length, over 9600 bytes, or audio arriving faster than real time allows. |
| `4010` | The server received no frame of any kind for 30 seconds.                                                   |
| `4200` | The session could not start. The close reason says whether to try again shortly.                           |

A session lasts at most 30 minutes, which ends it with `ended` reason `max_duration`. If the user says nothing, the agent checks in and then ends the session, with `ended` reason `server_ended`, unless the platform has recognized that the user described a medical emergency or self-harm, in which case silence leaves the session open.

## Client Rules

1. Check that the server selected the `auth` subprotocol and that `ready.protocol` is `amigo-voice-session.v1`. Close if not. Send nothing before `ready`.
2. Stream a frame every 20 ms from `ready` to the end: microphone while held, zeros otherwise.
3. Send `input.start` before the first audio frame of a press and `input.end` on release. Do not clear your playback queue on a press, so the opening notice is heard in full even when a press lands during it.
4. Clear the playback queue when `interruption` arrives.
5. Play the agent's audio through a small jitter buffer: hold the first chunk after a pause back by about 100 ms and schedule each chunk to start where the last one ends.
6. If the socket's send buffer holds more than about 2 seconds of audio, drop new frames instead of queuing stale speech.
7. On `1013` and `4003`, call start again. On `503` `no_voice_capacity`, retry once after `Retry-After`.

## Set Context on the Session

A voice session has a `call_sid`, so a trusted backend can bind verified values to it after it starts, with the same endpoint as for phone calls: `POST /agent/calls/{call_sid}/context`. The key must be declared in the service's `accepted_context`. The caller needs an admin or owner workspace API key, a higher role than the one that starts a session. Use the `call_sid` from the start response, and send the request after the app receives the `call_progress` frame: until the session is live the call is not found and the request returns `404`. Start-time context is not part of a session start request. See [Set Context on a Live Voice Call](https://docs.concurrence.com/developer-guide/guides/live-call-context.md) for the contract, including what a repeat or conflicting value returns.

## What Is Fixed

* The audio format is 24 kHz, 16 bit, mono in both directions. There is no other input rate and no transcoding.
* The turn mode is hold to talk.
* `gpt_live` supports multi-state Context Graphs in US workspaces without regional data residency. It also serves inbound phone calls, Playground, and test calls. Outbound calls, warm transfer, and operator takeover are not available.
* `eleven_realtime` also serves inbound phone calls, the Playground, and test calls. Outbound calls are not available. Phone calls support end call and cold transfer through the configured call controls. An app session does not dial a transfer destination. See [Eleven Realtime](https://docs.concurrence.com/developer-guide/platform-api/workspaces/voice-configuration.md#eleven-realtime).
* `grok_voice` also serves inbound phone calls, the Playground, and test calls, in workspaces without regional data residency. Outbound calls, warm transfer, and operator takeover are not available. See [Grok Voice](https://docs.concurrence.com/developer-guide/platform-api/workspaces/voice-configuration.md#grok-voice).

***

**You are here:** step 8 of 10 in [Learn by Building](https://docs.concurrence.com/developer-guide/guides.md#implementation-path), Run a Voice Session in Your App.

**Previous:** [Step 7: Build and Prove With Simulations](https://docs.concurrence.com/developer-guide/guides/build-and-prove.md). **Next:** [Step 9: Operate and Improve](https://docs.concurrence.com/developer-guide/guides/operate-and-improve.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.concurrence.com/developer-guide/guides/app-voice-session.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
