> For the complete documentation index, see [llms.txt](https://docs.concurrence.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.concurrence.com/developer-guide/platform-api/conversations/voice-agent.md).

# Voice Agent

Understand the real-time voice pipeline, context graph execution, tool calls, empathy adaptation, operator takeover, recordings, and post-call analysis.

The Concurrence voice agent powers real-time voice conversations. It handles inbound and outbound phone calls with speech understanding, text-to-speech, tool execution, safety monitoring, and context-graph or native/realtime runtime behavior. Supported service configurations can load projected patient context, emit source-attributed world events, and adapt each reply to the support the caller's words call for; those capabilities vary by runtime and configured tools.

{% hint style="warning" %}
**Reliability target.** This system handles healthcare scheduling calls where callers may be in distress, pain, or crisis. Optional intelligence features use best-effort fallbacks where the active runtime supports them. Failures in required telephony, speech, model, or runtime dependencies can still delay or end a call, so monitor call and session outcomes.
{% endhint %}

{% hint style="info" %}
**Voice settings and Classic API differences.** Voice settings (tone, speed, keyterms, sensitive topics, post-call flags) are configured at the workspace level. See [Workspaces, Voice Settings](https://docs.concurrence.com/developer-guide/platform-api/workspaces.md#voice-settings). Classic API offers [WebSocket voice streaming](https://docs.concurrence.com/developer-guide/classic-api/core-api/conversations/conversations-voice.md) for text-based apps. Platform API voice supports phone calls and, when configured, projected context and operator escalation.
{% endhint %}

## Audio Pipeline Architecture

The platform-managed phone runtime separates live input, reasoning, playback, and retained evidence. Selected world-model context and post-call evaluation depend on configuration, available data, and runtime support.

<figure><img src="https://1651797362-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YwM0LHtEGLt9S21OyC3%2Fuploads%2Fgit-blob-d3e4a957afbdfd2008f638484393b7120aa4c02f%2Fvoice-live-and-evidence.svg?alt=media" alt="Platform-managed voice processes caller input through Context Graph reasoning and controlled audio output. Retained call evidence supports review; separately configured evaluations have their own lifecycle."><figcaption><p>Post-call evaluation is separate. Evidence availability depends on runtime and configuration.</p></figcaption></figure>

### Layer 1: Signal Capture

**Speech-to-Text**: real-time streaming transcription. Three sources of domain vocabulary can improve recognition of configured terms:

1. **Service-level keyterms**: managed by workspace administrators, applied to all calls for that service.
2. **Workspace voice settings keyterms**: API-configurable per workspace (see [voice settings](https://docs.concurrence.com/developer-guide/platform-api/workspaces.md#voice-settings)).
3. **System defaults**: engineering-level fallback vocabulary.

All sources are merged and deduplicated per call. Configurable end-of-turn detection with tunable confidence thresholds determines when the caller has finished speaking, balancing responsiveness against cutting off mid-sentence.

### Layer 2: Intelligence

Each caller turn's words are read for the support they call for and assigned an empathy tier: Functional, Light Touch, Full Empathy, or Hold Space. The tier sets the empathy pause and hold, whether a filler is an empathy filler or a plain receipt, and the empathy guidance in the reply prompt. A turn whose read does not complete is treated as Functional. The observer stream can report the tier in an `empathy_classified` event. See [Empathy Tiers](https://docs.concurrence.com/developer-guide/platform-api/conversations/observer-events.md#empathy-tiers).

{% hint style="info" %}
**Recognized-speech barge-in.** On the platform-managed runtime, caller speech cuts the agent off only with recognized words and only when the active state's turn policy allows it, and never during the [call-opening notice](#call-opening). See [Barge-in](https://docs.concurrence.com/developer-guide/platform-api/workspaces/voice-configuration.md#barge-in) for the full rule and its settings. These gates reduce false triggers but do not eliminate recognition errors or background speech.
{% endhint %}

### Layer 3: Context Graph Engine

Each turn processes through a **two-stage LLM pipeline**:

1. **Navigator**: selects the next action or exit condition from the current context graph state, and can carry the chosen option's authored filler hint to cover the wait. If navigation fails or runs out of time, the agent stays in its current state.
2. **Engage LLM**: generates the caller-facing response, informed by the selected action, available conversation history, the turn's empathy guidance, ambient patient context, and available tools.

**Communication micro-behaviors**: when the standard engage template is used, it includes guidelines for micro-level conversational behaviors:

| Behavior                    | Description                                                                              |
| --------------------------- | ---------------------------------------------------------------------------------------- |
| **Speech rhythm mirroring** | If the caller speaks in short bursts, respond concisely. If conversational, match warmth |
| **Emotional name usage**    | Use the caller's name at moments of emotional significance, not mechanically             |
| **Pause injection**         | When delivering difficult information, pause naturally before the key detail             |
| **Pace inversion**          | When the caller is rushing, slow the pace with longer sentences and gentle transitions   |
| **Completion inference**    | When the caller trails off mid-sentence, acknowledge what they were trying to say        |
| **Natural laughter**        | Contextual laughter available for naturally warm moments, used sparingly                 |

### Layer 4: Audio Output

The engage LLM's text streams to the TTS engine for speech synthesis with **per-turn dynamic controls**:

* **Emotion**: selected through the [voice tone selection](#voice-tone-selection) path.
* **Speed**: from workspace [voice settings](https://docs.concurrence.com/developer-guide/platform-api/workspaces.md#voice-settings).
* **Volume**: from workspace voice settings.

When the active TTS provider supplies word timing metadata, the runtime can retain start/end offsets for transcript-to-audio alignment. Clients must not assume every provider or utterance includes complete word-level timestamps.

### Layer 5: Post-Call Intelligence

Terminal operational intelligence, retained analysis artifacts, and separately configured evaluations have different prerequisites and can be absent on an otherwise completed call.

**Transcripts.** The platform does not re-transcribe calls after they end or score transcript accuracy. Read the live transcript and available recording. Do not wait for verification fields or manufacture a zero accuracy score. Terminal operational summaries are available. There is no separate transcript-only sentiment, topic, or intent analysis.

**Quality analysis**: `call_analysis` is an optional compatibility field, and the `post_call_analysis_enabled` setting does not start a recording-analysis job. Use [Call Trace Analysis](https://docs.concurrence.com/developer-guide/platform-api/safety/call-trace-analysis.md), [Voice Judge](https://docs.concurrence.com/developer-guide/platform-api/safety/voice-judge.md), and [Production Evals](https://docs.concurrence.com/developer-guide/platform-api/safety/production-evals.md) for their separately documented outputs and prerequisites.

#### Call Intelligence Persistence

The platform-managed runtime records a structured intelligence summary at call end, independently of recording analysis. It captures operational signals that are not available from transcript scoring alone.

Fully analyzed call intelligence records contain:

| Field                            | Type          | Description                                                                                                                  |
| -------------------------------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `quality_score`                  | float or null | Rule-based operational quality score (0-100), penalty-based                                                                  |
| `latency_summary`                | object        | Engine response time (avg/p50/p95) and audio TTFB (avg/p50/p95). `silence_ratio` is null because call silence isn't measured |
| `conversation_summary`           | object        | `turn_count`, `states_visited_count`, `unique_states`, `loop_count`, barge-in count, completion reason                       |
| `tool_summary`                   | object        | Total calls, success/failure counts, failure rate, per-tool breakdown                                                        |
| `operator_summary`               | object        | Whether the call escalated and, when it did, whether an operator was active                                                  |
| `risk_summary`, `safety_summary` | object        | Compatibility fields that are empty in the current runtime. They are not active risk-scoring or safety-triage outputs        |
| `completion_reason`              | string        | Why the call ended (hangup, terminal state, silence, etc.)                                                                   |
| `final_state`                    | string        | Last context graph state at call end                                                                                         |

Some external voice runtimes produce only a terminal call-intelligence envelope, with `quality_score` null and the analysis summaries empty. That does not mean their conversation history is empty: the `gpt_realtime` path separately retains transcripts and tool calls when persistence succeeds. See [Call Record & Persistence](#call-record-and-persistence).

**Quality score penalties:**

| Signal        | Threshold               | Penalty    |
| ------------- | ----------------------- | ---------- |
| High latency  | p95 audio TTFB > 1000ms | -5 to -15  |
| Barge-ins     | > 2                     | -5 to -15  |
| Agent loops   | > 0 revisited states    | -10 to -20 |
| Escalation    | Any                     | -10        |
| Tool failures | Failure rate > 5%       | -5 to -15  |

#### Call Intelligence Endpoints

The call intelligence endpoint exposes the persisted terminal summary for a completed call:

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/calls/{call\_id}/intelligence" method="get" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

Returns 404 if the call or intelligence data is not found.

**Recognition review**: compare the live transcript with available recordings and validate recognition keyterms explicitly. The platform does not emit an `stt_suggestions` completion payload, so clients should not wait for one.

## How Calls Work

Supported phone-call flows use a **conference architecture**, a multi-party audio bridge that lets the caller, AI agent, and optionally a human [operator](https://docs.concurrence.com/developer-guide/platform-api/conversations/operators.md) participate simultaneously.

Call identifiers returned as `call_sid` are opaque platform handles, distinct from run IDs and individual carrier call-leg identifiers. Preserve them unchanged across observer, recording, and supported call-scoped operations; see [Calls](https://docs.concurrence.com/developer-guide/platform-api/conversations/calls.md). Inbound phone identity can be absent: valid caller numbers are normalized to E.164, while anonymous, withheld, invalid, or non-phone origins remain unidentified. Handle missing identity before relying on caller lookup or automatic [channel context](https://docs.concurrence.com/developer-guide/guides/conversation-context.md#reserved-channel-identity).

### Inbound Call Preparation

Supported inbound telephony paths can reduce startup delay through best-effort **parallel pre-warming**: engine initialization, greeting preparation, patient-context resolution, and the agent connection can begin while the phone is ringing. Completion before pickup is not guaranteed.

<figure><img src="https://1651797362-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YwM0LHtEGLt9S21OyC3%2Fuploads%2Fgit-blob-6eab98d67c651c5ed85502e70af5c9f85481ce2e%2Fvoice-call-lifecycle.svg?alt=media" alt="A phone call resolves service and runtime, opens with the AI and recording notice, follows runtime-specific interaction and handoff controls, and ends with independently available retained evidence."><figcaption><p>Preparation is best-effort. Greeting, interruption, and ending behavior differ by runtime.</p></figcaption></figure>

The conference name is available at webhook time in the standard telephony implementation, allowing the runtime to begin creating the agent leg before pickup. Whether the agent connection, context, and greeting are ready at answer time depends on ring duration and downstream dependencies.

{% hint style="info" %}
**Pre-warm is best-effort.** Startup work that has not finished by pickup continues through the ordinary initialization path. Time to first audio varies by provider, runtime, configuration, and dependency health. Pre-warming is not a latency guarantee.
{% endhint %}

### Call Opening <a href="#call-opening" id="call-opening"></a>

Platform-managed and `gpt_realtime` calls open with a notice that discloses AI use and call recording to the caller. A call whose voice speaks Spanish hears the same notice in Spanish, and other languages hear the English wording. The notice is part of the call's transcript.

<figure><img src="https://1651797362-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YwM0LHtEGLt9S21OyC3%2Fuploads%2Fgit-blob-556febb7f93be8b196b32bbc07dc651c7fcd54f0%2Fcall-opening-by-runtime.svg?alt=media" alt="On the platform-managed runtime, the AI and recording notice plays before the greeting or first reply. On gpt_realtime, an inbound call hears it before the greeting, and a Playground or simulated call hears it after the caller&#x27;s first turn."><figcaption><p>Where the notice plays depends on the runtime and the kind of call. A platform-managed outbound call opens once the callee answers. On <code>gpt_realtime</code>, a caller who describes an emergency during the opening hears the emergency script first, a notice it cut off plays again, and the greeting is dropped.</p></figcaption></figure>

* **Platform-managed runtime.** The notice plays ahead of the call's first speech, whether that is the greeting or the first reply on a call where the caller speaks first. Caller speech does not cut it off, and barge-in waits until it has played. On an inbound call, what the caller says at pickup is held while speech recognition connects and is transcribed once it is ready, so an opening "hello" becomes a turn. An outbound call opens once the callee answers. See [Outbound Call Flow](#outbound-call-flow).
* **`gpt_realtime`.** The model speaks the notice as the call's first response. An inbound call hears it before the greeting, and the greeting then plays in full. Anything the caller says meanwhile is answered after it. A Playground or simulated call hears it after the caller's first turn, then the answer to that turn. If the caller describes an emergency during the opening, the [emergency script](#realtime-emergency-script) plays first, a notice it cut off plays again in full, and the greeting is dropped.

If the notice cannot be rendered, for example after a speech-synthesis error, the call continues without it. Remove any AI or recording notice from your own greetings and prompts so callers do not hear it twice, and check greeting timing in a test call. Text conversations, on every channel, do not open with this notice. A call that takes the [safe exit](#graceful-degradation) before the caller has heard all of the notice hears the notice first. A service that declares both English and Spanish opens a call on a realtime family other than `gpt_live` with the notice in each language. See [Languages a Service Serves](#languages-a-service-serves).

### Languages a Service Serves <a href="#languages-a-service-serves" id="languages-a-service-serves"></a>

A service serves the languages in its [`declared_languages`](https://docs.concurrence.com/developer-guide/platform-api/workspaces/services.md#key-concepts), or only its agent's default language (`identity.default_spoken_language`) when it declares none. A voice call starts in the agent's default language, and a caller's language replaces it only when the service serves that language. A save that leaves the agent's default language out of `declared_languages` returns `422`.

#### Serve English and Spanish

Declare `en` and `es` and give the agent English as its default language. On `gpt_realtime`, `eleven_realtime`, `gpt_live`, `grok_voice`, and `phonic`, the call then runs a bilingual session: it starts in English and follows the caller's clear choice of English or Spanish. Each family applies the session in its own way:

| Family                                      | What the session does                                                                                                                                                             |
| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `gpt_realtime`                              | The call's language instructions are the bilingual ones, and, while `realtime.transcription.language` is unset, the caller's speech is transcribed expecting English and Spanish. |
| `grok_voice`, `eleven_realtime`, `gpt_live` | The model is not told to answer in one language, so it can answer in either. `grok_voice` sets no transcription language hint unless the service sets `language_hint`.            |
| `phonic`                                    | The session starts in English and by default takes Spanish on request.                                                                                                            |

A service that declares both languages also opens its calls on these families, except `gpt_live`, with the AI and recording notice in each language, in the order declared. The Spanish notice is followed by an invitation to speak Spanish when the model says the opening. An inbound call then hears a greeting in English that names the agent and its organization, by default tells the caller what to do in a medical emergency, says when the service's [`forwarding.hours`](https://docs.concurrence.com/developer-guide/platform-api/workspaces/voice-configuration.md#transfer-hours) show it closed, and asks how it can help. On an outbound call the opening leaves out the greeting, and the model greets or answers after it.

On `grok_voice` and `phonic`, the notices play from prerecorded audio by default, and the model then greets in its own voice in place of the invitation and the authored greeting. A `gpt_live` call and a platform-managed call open with one notice.

Test a call in each language on the family you choose before you move a production service. Model behavior differs between families.

#### What a Bilingual Service Does Not Cover

* A service whose agent's default language is Spanish keeps a Spanish-only session on the realtime families, even when it serves English too.
* A language pair other than English and Spanish runs a realtime session in the agent's default language alone.
* A third declared language is not followed on the realtime families: the session follows English and Spanish only. The opening has a notice only in English and Spanish.
* The platform-managed runtime has its own language rule. See [Language Selection](https://docs.concurrence.com/channels/voice/audio-pipeline#language-selection).

### Outbound Call Flow

Outbound calls start through the [Calls API](https://docs.concurrence.com/developer-guide/platform-api/conversations/calls.md).

**Outbound preparation**: the standard outbound path prepares the first agent audio before dialing the callee. If that preparation fails or exceeds its budget, the request fails instead of falling through to an unprepared dial. Provider connection and playback can add latency after answer, so this is not a time-to-first-audio guarantee.

**Waiting for the callee**: on the platform-managed runtime, the agent says nothing while the phone rings. When the callee answers, the call opens with the [notice](#call-opening) and the greeting. A voicemail that picks up counts as an answer, so the notice and greeting play to it. Keep the greeting fit to leave on a voicemail. A dial that is busy, goes unanswered, or fails ends the call before the agent speaks.

### Conference Architecture

<details>

<summary>Conference architecture: telephony details</summary>

The conference architecture supports multiple simultaneous audio participants with independent per-participant streams:

Caller, agent, and an optional phone operator participate in the conference. Supported transcription paths attribute human speech to the caller or operator. Operator mode determines whether the agent continues or yields control. It is separate from speech recognition.

| Participant  | Role                              | Audio Transport         | STT                              |
| ------------ | --------------------------------- | ----------------------- | -------------------------------- |
| **Caller**   | Person who called or was called   | PSTN                    | Dedicated per-participant stream |
| **Agent**    | AI voice agent                    | Bidirectional WebSocket | Main session STT                 |
| **Operator** | Human monitor/takeover (optional) | PSTN                    | Dedicated per-participant stream |

**Three-party speaker resolution**: when multiple parties are on the call, speaker attribution uses a priority chain: operator STT → caller STT → default (caller). Recorded turns can carry `speaker_id` and `speaker_role` to support transcript attribution. Clients should tolerate missing or ambiguous metadata when a stream or persistence step is incomplete.

</details>

## Context Graph Engine

The platform-managed voice runtime executes a **Hierarchical State Machine** loaded from the service's version set. Each call maintains isolated conversation state. The state types and navigation flow below describe this runtime.

`gpt_realtime` accepts one action state or a graph of 2 to 12 action states with unique names and valid exit targets. Both initial-state fields on a multi-state graph must name the first state. The model reasons from the current state's instructions and tools. It requests a move through `change_state`, whose available destinations are the current state's exits. The platform applies the move and waits for the voice provider to acknowledge the new instructions and tools before continuing. A model can miss a move and stay in the current state. Entering a terminal state does not itself end the call.

`eleven_realtime` requires exactly one action state. `gpt_realtime` runs the service's enabled [dynamic behaviors](https://docs.concurrence.com/developer-guide/platform-api/workspaces/dynamic-behaviors.md#services-on-gpt-realtime), and `eleven_realtime` accepts none. Neither family accepts typed `guardrails` on an action state, and `eleven_realtime` also refuses a tool call spec that sets `navigate_on_completion`. A save that would pair a family with an unsupported graph returns `422`. See [Session Provider](https://docs.concurrence.com/developer-guide/platform-api/workspaces/services.md#session-provider). An unchanged pairing is not rechecked on save. If preparing a call or applying a move fails, the call takes the [safe exit](#graceful-degradation). Outbound `gpt_realtime` calls require a single action state. The provider selects from authorized tools without the platform-managed navigator. Operator takeover, session event injection, and [approval-gated tools](#execution-model) are not supported on this path.

When a `forward_call` is emitted by a response that the caller barged in on, the realtime turn owner can still dispatch that transfer if its arguments arrived as a complete JSON object, treating the transfer as if it had landed just before the barge-in. Other tool calls from an interrupted response - including reads, writes, and `end_call` - are not executed and return a not-executed result instead, and a superseded tool result is not spoken after the caller interrupts. A transfer whose arguments are cut off or empty is also not executed. Separately, caller speech while a transfer is still playing its announcement cancels the transfer. This behavior is specific to the `gpt_realtime` runtime.

### State Types

| State Type              | Purpose                                                                               | LLM Call?                                 |
| ----------------------- | ------------------------------------------------------------------------------------- | ----------------------------------------- |
| **ActionState**         | Produces user-facing responses, eligible tool calls, and exit-condition evaluation    | Yes (engagement and navigation as needed) |
| **DecisionState**       | Selects among authored exits without producing a user-facing turn                     | Yes (navigation only)                     |
| **AnnotationState**     | Injects authored internal guidance and advances to its fixed next state               | No additional model call                  |
| **DataCollectionState** | Collects declared fields across turns, with turn limits and optional Surface fallback | Yes (engageable)                          |

### Per-Turn Flow

<figure><img src="https://1651797362-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YwM0LHtEGLt9S21OyC3%2Fuploads%2Fgit-blob-86aba4a2ea492f10c18ca99365ac66979b524299%2Fvoice-timeline-blue.svg?alt=media" alt="A platform-managed voice turn forms input from recognized speech, navigates the Context Graph, generates a response with eligible tools, and plays speech under voice controls. Filler is optional and playback is distinct from tool completion."><figcaption><p>Platform-managed voice. Tool execution, speech playback, and retained evidence have separate boundaries.</p></figcaption></figure>

The navigator handles multi-state traversal automatically. Decision and annotation states can resolve without user interaction before landing on an action state for the engage LLM.

**Navigator resilience**: a navigation that fails or runs out of time leaves the agent in its current state, and the agent answers the turn from there. The runtime never takes an option that navigation did not choose.

### Action State Extensions

Action states support three optional extension fields for asynchronous workflows:

| Field                   | Type   | Description                                                                                                                                                                                                              |
| ----------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `wait_for`              | string | Pause navigation until an async condition clears. Values: `surface_submission`, `human_approval`.                                                                                                                        |
| `channel_overrides`     | object | Per-channel overrides keyed by channel kind (`voice`, `sms`). Each override can set `objective`, `action_guidelines`, and `progress` (a [progress hint](#tool-wait-progress-hints) for tool waits in that state).        |
| `surface_spec_template` | object | Surface spec auto-created on state entry. Uses the [Surfaces](https://docs.concurrence.com/developer-guide/platform-api/conversations/surfaces.md) field schema. The entity ID defaults to the session's primary entity. |

**Wait conditions**: while a state waits, the engine skips context graph navigation on subsequent turns and the agent is told not to move on to new topics. A `surface_submission` wait also tells the agent not to collect new information and lets it answer questions and reassure the patient, and a `human_approval` wait tells it to say the information is under review and stay patient and supportive. This is guidance to the model, not a deterministic restriction. A wait ends only on the event it names. A submitted form ends a `surface_submission` wait and leaves a `human_approval` wait in place, and a review approval ends a `human_approval` wait. A `surface_submission` wait ends when a form the conversation sent after the wait began is submitted. If the conversation has sent no form since the wait began, the submission of any form it sent ends the wait, including one sent before the wait began.

**Channel overrides**: the `channel_kind` is set on the engine session (`voice` for calls, `sms` for text sessions). Prompt rendering merges the channel-specific objective and guidelines into the engage prompt. For voice, the override can also carry a `progress` hint that shapes how the agent covers tool waits in that state (see [Tool-Wait Progress Hints](#tool-wait-progress-hints)).

**Surface templates**: on state entry, if the new state has a `surface_spec_template`, the engine requests a surface through the Platform API and tracks the `surface_id` when creation succeeds. This makes surface creation configuration-driven rather than dependent on a model-selected tool call.

`wait_for` and `surface_spec_template` act where the engine navigates, on the platform-managed runtime and in text conversations. In text conversations only a web conversation is told that a form was submitted or approved, so a wait in an SMS, WhatsApp, or email conversation is not released by either event. On `gpt_realtime`, including multi-state graphs, `preload`, `wait_for`, `surface_spec_template`, `turn_policy`, and `channel_overrides` have no effect. A successful graph save does not make these fields apply.

### Terminal State & Auto-Hangup

When the platform-managed runtime reaches the Context Graph's configured terminal state, it speaks a final response and then ends the call:

1. Navigation lands on the terminal state.
2. The agent speaks its closing response.
3. The runtime waits for that audio to finish playing, and for any tool still running.
4. It ends the call through telephony.

`gpt_live` ends the call the same way when navigation lands on the terminal state: the closing reply plays, then the call ends. On `gpt_realtime`, entering a terminal state does not itself end the call.

`terminal_state` is optional on a Context Graph version. A version saved without it has no terminal state, and navigation never ends the call on any voice family, and the call ends another way, such as the caller hanging up, the silence check-ins running out, or the [safe exit](#graceful-degradation). On a single-state graph, leave `terminal_state` unset unless the call should end after the first answered turn: naming the only state as terminal ends the call once navigation first lands there, on the platform-managed runtime and on `gpt_live`.

When a caller's turn is recognized as a request for a person, navigation does not land on the terminal state on that turn: the agent stays in its current state and answers the request (see [Safety & Escalation](#safety-and-escalation)).

**Talking over the goodbye**: on both runtimes, a caller who starts speaking before the goodbye's hangup goes out holds it until they finish. If what they said is only a short closing, such as "thanks, bye", the call ends as planned with no reply. Anything else keeps the call open, and the agent answers it. Speech with no words in it ends the call at the next silence.

**After an emergency**: once the caller describes a medical emergency or self-harm, in English or Spanish, the agent doesn't end the call, on either runtime, and on `gpt_realtime` it speaks the [emergency script](#realtime-emergency-script) first. On the platform-managed runtime, navigation doesn't land on the terminal state for the rest of the call, and a call already there goes back to the state it was in before the goodbye. On `gpt_realtime`, the model's `end_call` waits until the caller's last words have been checked, and it is refused once the caller has described an emergency. The last silence below leaves the line open. The call then ends when the caller hangs up or the call reaches its duration limit, unless a `gpt_realtime` service is set to end it after the script for a medical emergency. See [Realtime Emergency Script](#realtime-emergency-script). The [safe exit](#graceful-degradation) can still end it, as can an operator who has taken the call on the platform-managed runtime. Recognition can miss a phrasing, so author the graph's emergency path to stay with the caller too.

**Silence**: on both voice runtimes, 10 seconds of quiet after the agent finishes speaking brings a check-in, one short new sentence such as asking whether the caller is still there. The next 10 seconds of quiet brings a second check-in, and the 10 seconds after that ends the call without a closing line, unless the caller has described an emergency. When the caller speaks, the count starts over. On the platform-managed runtime, a caller who is speaking as that hangup comes due holds it the same way as one who talks over a goodbye.

<figure><img src="https://1651797362-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YwM0LHtEGLt9S21OyC3%2Fuploads%2Fgit-blob-e754369920b3b458580b13ca82b1193e7b1607e7%2Fsilence-check-ins.svg?alt=media" alt="On both voice runtimes, 10 seconds of quiet after the agent speaks brings a check-in, another 10 seconds brings a second one, and the next 10 seconds ends the call, unless the caller has described an emergency. Caller speech starts the count over, and progress updates cover a tool that is still running."><figcaption><p>The same sequence runs on the platform-managed runtime and on <code>gpt_realtime</code>. After a caller describes an emergency, the last silence leaves the line open.</p></figcaption></figure>

While a tool is still running, the platform-managed runtime gives a short progress update in place of a check-in, for a limited number of updates, and `gpt_realtime` holds check-ins until the tool finishes and plays [progress lines](#tool-wait-progress-hints) meanwhile. If the caller starts speaking before a check-in plays, the check-in is dropped. Silence counts for nothing while a transfer is under way, through its hold, dial, and briefing. On the platform-managed runtime it also counts for nothing while an operator has the call, and the count starts over when the caller is handed back.

**Session shutdown behavior**: the standard shutdown handlers stop both the session and audio speaker for hangup, STT failure, WebSocket disconnect, and terminal-state paths. Clients should rely on the reported terminal outcome rather than assuming cleanup timing.

## World Model Integration

The voice agent connects to the workspace's [world model](https://docs.concurrence.com/developer-guide/platform-api/data-world-model.md) through three data channels. See [Storage and Projection Boundaries](https://docs.concurrence.com/developer-guide/platform-api/data-world-model.md#storage-and-projection-boundaries) for the distinction between durable events and asynchronously projected entity state.

Selected ambient context, explicit queries, and supported extraction are separate data paths. Caller-facing protected record access requires identity verification. Captured observations retain provenance and become readable through asynchronous projection; they do not automatically authorize an external write.

### Channel 1: Ambient (Pushed)

Supported runtimes can inject selected ambient data without an explicit agent lookup. Patient fields load only after [caller verification](#caller-verification). Available fields depend on verification, projected state, service configuration, and successful prompt construction, and can include:

* **Patient demographics**: name, DOB, MRN, phone, email, address.
* **Clinical context**: active conditions, medications, allergies (filtered to text-only for LLM consumption).
* **Upcoming appointments**: with patient entity references for cross-referencing.
* **Insurance coverage**: active plans and subscriber info.
* **Location context**: clinic details, available appointment types, hours (resolved from the inbound phone number).

**Design principle: ambient over queried.** If the LLM will almost certainly need this data, push it into context. Don't make it ask. A voice agent that already has the patient's insurance in context doesn't need to dispatch a tool call to look it up.

### Caller Verification <a href="#caller-verification" id="caller-verification"></a>

A caller who matches a record by phone number, or a session opened with an `entity_id`, starts unverified. The agent is told a record may match, but the record's demographics and clinical context stay out of its context. They load after the caller's stated full name and date of birth match exactly one candidate record through the `world_confirm_patient_identity` tool. A failed attempt does not say which field was wrong, and repeated failures lock verification for the rest of the conversation. Text sessions restore the verified or locked state from the conversation history on each turn.

<figure><img src="https://1651797362-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YwM0LHtEGLt9S21OyC3%2Fuploads%2Fgit-blob-dc33c96c0185d7bb91e736d96fe0b6c8ccba8d5f%2Fcaller-verification-states.svg?alt=media" alt="A caller matched by phone number or entity_id starts unverified, and built-in patient reads return verification_required. A full name and date of birth that match one record load the patient context, and repeated failures lock verification for the conversation."><figcaption><p>The check covers the built-in patient-record tools on voice and text conversations.</p></figcaption></figure>

Add `world_confirm_patient_identity` to the Context Graph state that confirms identity, and have that state ask for the caller's full name and date of birth. Until a graph offers the tool, the agent works without the record. The tool's result says whether the caller verified, and after a failed attempt, how many attempts remain. Sessions anchored to an authenticated operator skip this check.

A service whose workflow never needs the caller's record can turn the check off with `platform_caller_verification: false`. The platform never asks its callers to verify, and `world_confirm_patient_identity` is never offered, even where the graph lists it. The record never reaches the agent's context, and the agent is told not to greet the caller by name or share anything from a patient record. Simulations, test sessions, and operator-anchored sessions aren't affected by the setting. A service whose calls go to a party that is not the patient, such as a payer line, can set `platform_caller_verification` to `false` so the agent does not ask that party to verify.

The record tools follow the same rule:

* **Before verification,** every built-in patient-record read, such as `world_patient_lookup`, `world_appointment_lookup`, `world_get_medications`, and the typed clinical lookups, returns status `verification_required` with no record, and `world_confirm_patient_identity` is offered next to it so the agent can verify and retry. Operator-anchored, simulation, and test sessions are not gated.
* **After a lock,** those reads return status `verification_locked` for the rest of the conversation. The agent is told not to ask for the caller's details again, and to offer to connect them with staff or help with something that needs no patient record.
* **After verification,** those reads return the caller's own record, or a household member's whose record names the caller as guardian or under a power of attorney. A read for anyone else returns status `not_caller_record`. A relationship the caller states does not widen this.
* **With verification off,** those reads return status `verification_off` for the whole conversation. On voice, the reply-check correction for record details is "Sorry, I can't share anything from a patient's record here.", or its Spanish version, and asks the caller for nothing. The service's integration tools run as usual, unless the service names one that verifies its callers (`caller_verification_endpoint_id`).
* **Scope.** The check covers the built-in `world_*` record tools. Patient reads through your own [integration](https://docs.concurrence.com/developer-guide/platform-api/integrations.md) endpoints are not gated by it; the agent is instructed not to share another person's details from any tool, which is model guidance, so test other-person requests on graphs that read records through integrations.

#### Unverified integration results

`allow_unverified_integration_results` (default `false`) is a bypass for a service whose integration endpoints return data the agent has to read back to any caller. When it is `true`, results from the service's integration endpoints, reads and writes, do not count as a patient record for the unverified-caller reply check (`output_checks.phi_unverified`, which runs only on a service that sets it to `cut`). The agent can read those results back to a caller whether or not the caller verified.

The following stay enforced with the bypass on:

* Writes are held to every other check.
* The built-in patient-record tools still refuse until the caller verifies.
* Data queries and skills still count as patient records for the reply check.

Only a Concurrence platform administrator (the `platform:admin` scope) can turn the bypass on. A workspace owner or admin cannot, because the bypass lets integration results reach an unverified caller. A request that turns it on without the scope is refused with `unverified_results_need_platform_admin`. Turning it off needs `Service.update`, and every change to it is audited.

A service cannot combine the bypass with `caller_verification_endpoint_id`. A save that sets both is refused.

A date of birth can be spoken in words, such as "May fifteenth nineteen ninety". On a graph that branches on `world_patient_lookup`, a caller who verifies follows the found branch, and a caller whose name and date of birth match no record follows the not-found branch, without the refused lookup revealing anything. `world_patient_create` on a medical record number already on file does not say whose record it is.

### Channel 2: Queried (Pulled)

Data that can't be ambient because the search space is too large. The agent calls [built-in clinical tools](#built-in-clinical-tools) to retrieve specific information.

**Key simplification**: standard queried tools return human-readable results rather than database internals. Slot search can return doctor names and times while the runtime carries the selected scheduling identifiers into a later booking request, so raw template IDs and slot UUIDs do not need to be exposed to the LLM.

### Channel 3: Extracted (Captured)

Supported voice runtimes can extract structured observations such as insurance details, contact information, and preferences from conversation evidence. When extraction and event publication succeed, those observations enter the world model without a separate conversational tool call. Extraction is model-generated, can be incomplete, and is not enabled identically on every runtime.

Extracted data uses a source-appropriate confidence class below authoritative records. Explicit write tools remain the controlled path for supported high-stakes mutations. Extraction complements those tools. It does not prove that every mentioned fact was captured or externally delivered.

### Confidence-Gated Writes

World-model observations written by the voice agent during calls carry a **confidence level**: a source-class ranking, not a calibrated probability of truth. Confidence is one input to the policies that govern external synchronization.

| Confidence | Level          | Source Class                                          |
| ---------- | -------------- | ----------------------------------------------------- |
| 0.0        | Rejected       | Human-rejected or system-invalidated                  |
| 0.3        | Agent raw      | Raw voice agent inference (unverified)                |
| 0.5        | Self report    | Patient-confirmed or patient-submitted data           |
| 0.7        | Verified       | Verified or EHR-ingested data                         |
| 0.8        | EHR trusted    | Selected trusted clinical data                        |
| 0.9        | High           | Operator-verified or high-quality source              |
| 0.95       | Human approved | Named registry class, not assigned by proposal review |
| 1.0        | Authoritative  | Manual entry or authoritative API                     |

**Confirmation-gated confidence**: the agent chooses a confirmation level per write based on how the information surfaced in conversation - `confirmed` (the caller explicitly confirmed the fact, written at self-report confidence), `mentioned` (mentioned in passing, written at agent-raw confidence), or `inferred` (inferred from context, written below agent-raw confidence).

**External delivery**: these observations stay in the world model; an external change needs its own integration call. Confidence is available as source evidence for workflows that act on it.

### Patient Safety Isolation

Supported entity-anchored write tools enforce a **write scope** for the session and reject a target outside the resolved patient anchor. This is an authorization guard, not proof that the selected field, value, or patient anchor is clinically correct. Write tools also have **in-flight deduplication**: if the model re-calls the same write tool with identical parameters while the first call is still executing, the concurrent duplicate is suppressed. Once the first execution finishes, the same parameters can execute again, except an integration write whose outcome is unknown, which is not sent again in that conversation (see [Execution Model](#execution-model)). This guard is not general request idempotency or an exactly-once guarantee.

## Voice Adaptation

The empathy tier and the conversation can influence prompt guidance, TTS tone, filler behavior, and empathy pacing. These effects are best-effort and depend on the active runtime and speech provider. Workspace and service settings can override or disable them. For the conceptual treatment, see [Audio Pipeline](https://docs.concurrence.com/channels/voice/audio-pipeline#situation-response-adaptation).

### Voice Tone Selection

Navigation can pick the tone of the next spoken reply from a fixed tone set, based on the conversation. See [Turn-Level Emotion Selection](https://docs.concurrence.com/developer-guide/platform-api/workspaces/voice-configuration.md#turn-level-emotion-selection). When navigation picks no tone, the standard voice-context path chooses one in this order:

1. If the current action matches a configured sensitive topic, select `sympathetic`.
2. If workspace `tone` is configured, it explicitly overrides the computed tone.

### Filler Speech

Fillers can cover processing latency. The system uses **principle-based guidance** rather than relying only on hardcoded phrase lists, generating contextually appropriate fillers from the turn's empathy tier, the current action, and expected latency.

**Three-layer filler generation:**

| Layer                  | When                                    | What It Controls                                                                     |
| ---------------------- | --------------------------------------- | ------------------------------------------------------------------------------------ |
| **Latency adaptation** | When filler generation runs             | Filler length can reflect expected processing time (typically 2-4 words)             |
| **Empathy**            | At the Light Touch tier and above       | The filler is an empathy filler (warmer, acknowledging) rather than a plain receipt. |
| **Action context**     | When an action description is available | The action description can steer a filler toward what the agent is about to do       |

**Per-action filler hints**: context graph actions can include optional PM-configured filler suggestions. In non-deterministic mode, the LLM sees them as suggestions alongside contextual guidance, not commands. Explicit deterministic phrases follow the separate contract below.

**Suppression**: the service's `silent` [filler style](https://docs.concurrence.com/developer-guide/platform-api/workspaces/voice-configuration.md#fillers) turns autonomous fillers off.

#### Tool-Wait Progress Hints

When a tool is running, the agent narrates the wait based on a `ProgressHint` declared on the state's channel override and/or on the individual tool binding. The hint describes the *shape* of the wait, not a phrase list - the engine composes the actual utterance from tool semantics, the turn's tone, and conversation context.

| Field                 | Type                                                                                  | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| --------------------- | ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `mode`                | `"auto"` \| `"silent"` \| `"backchannel"` \| `"verbal"`                               | How the agent covers the wait. `auto` lets the engine decide from class and latency. `silent` drops filler text. `backchannel` emits a single token ("Mm."). `verbal` produces a contextual filler.                                                                                                                                                                                                                                                                                            |
| `progress_class`      | `"lookup"` \| `"write"` \| `"external_call"` \| `"compute"` \| `"multi_step"` \| null | Semantic category of the work the tool is doing. Drives the templated language the engine picks for retries.                                                                                                                                                                                                                                                                                                                                                                                   |
| `expected_latency_ms` | integer (0-60000) \| null                                                             | How long the tool is expected to take. Used to choose filler length and pacing.                                                                                                                                                                                                                                                                                                                                                                                                                |
| `trigger_delay_ms`    | integer (0-30000) \| null                                                             | How long to wait after a tool starts before playing the first filler. Set to `0` for immediate playback. When unset, the delay is calculated from `expected_latency_ms`. When unset and deterministic phrases are configured, the first filler fires immediately.                                                                                                                                                                                                                              |
| `interval_ms`         | integer (100-30000) \| null                                                           | Milliseconds between subsequent fillers. Overrides the global progress interval for this hint.                                                                                                                                                                                                                                                                                                                                                                                                 |
| `deterministic`       | boolean (default `false`)                                                             | Whether `phrases` play verbatim. `true`: scripted - the engine plays the phrases exactly as written, in order. `false`: the engine picks - phrases are hints.                                                                                                                                                                                                                                                                                                                                  |
| `phrases`             | array of strings (1-10 items) \| null                                                 | Ordered filler phrases, each at most 30 words. With `deterministic: true`, phrases play verbatim in order, cycling back to the start for attempts beyond the list length. With `deterministic: false` (the default), phrases are used as hints for early attempts - attempt N plays `phrases[N-1]` while within the list, and later attempts fall back to class templates, vocabulary, and retry fillers. `backchannel` mode preempts phrases. When set, `phrases` supersedes `custom_phrase`. |
| `custom_phrase`       | string (max 500 chars) \| null                                                        | Operator-authored filler utterance for the first filler attempt. In `auto` mode: requires `expected_latency_ms` >= 4000 and `progress_class` to be set. In `verbal` mode: always honored (word cap still applies). Ignored when `phrases` is set. Subsequent attempts use class templates.                                                                                                                                                                                                     |

**Placement**: `progress` can be set on `channel_overrides[channel].progress` (state-level default for that channel) and on each entry of `action_tool_call_specs` or `exit_condition_tool_call_specs` (per tool). Per-tool fields merge field-wise over the channel override, so a state can declare the default wait shape once and individual tools only override fields that actually differ.

**Retry narration**: when a tool retries, the engine uses deterministic attempt-aware templates keyed on `progress_class` - acknowledgement on attempt 1, brief apology on attempt 2, a "still working" update from attempt 3 - so retry audio stays off the LLM hot path and latency stays bounded. With `deterministic: true`, the engine skips templates and plays the scripted phrases in order, cycling through the list. With `deterministic: false`, phrases cover early attempts and templates take over for attempts beyond the list.

**Deterministic vs non-deterministic**: scripted flows (demo flows, regulated disclosures) that need exact control over what the agent says during tool waits must set `phrases` with `deterministic: true`. Deterministic fillers require the tool binding's `execution: "background"` - write-time validation rejects a blocking binding combined with `deterministic: true`. Use `custom_phrase` + `progress_class` when you want to steer the first filler but let the engine handle retries with contextual templates.

**Both runtimes**: a `gpt_realtime` caller waiting on a tool hears the same progress lines on the same clock, timed by the hint as above or by the service's `progress_interval_ms` (3 seconds by default), from the graph's phrases, the service's `progress_vocabulary`, or the built-in lines. A `silent` hint stays silent. Each line is said as written rather than generated. A multilingual agent says each built-in line in English, then Spanish. When a [workspace data query](https://docs.concurrence.com/developer-guide/platform-api/functions/workspace-data-queries.md) times out on an English agent's `gpt_realtime` call and is the only result the turn has to report, the agent says an authored line, "Sorry, that took longer than expected and I wasn't able to finish just now. Could you try asking again?", in place of its own wording about the timeout.

### Proactive Intelligence

The system detects emotionally sensitive topics from the current context graph action **before the caller shows distress**:

```mermaid
%%{init: {"theme":"base","themeVariables":{"fontFamily":"Inter, sans-serif","fontSize":"16px","background":"#FAF8F6","primaryColor":"#F1EAE7","primaryTextColor":"#1A1817","primaryBorderColor":"#D7D2D0","secondaryColor":"#FDFCFB","tertiaryColor":"#F6F2F0","lineColor":"#575452","textColor":"#3D3937","clusterBkg":"#FDFCFB","clusterBorder":"#E8E2DF","edgeLabelBackground":"#FAF8F6","actorBkg":"#F1EAE7","actorTextColor":"#1A1817","actorBorder":"#D7D2D0","actorLineColor":"#575452","signalColor":"#575452","signalTextColor":"#3D3937","labelBoxBkgColor":"#F1EAE7","labelBoxBorderColor":"#D7D2D0","labelTextColor":"#3D3937","loopTextColor":"#3D3937","noteBkgColor":"#F6F2F0","noteBorderColor":"#D7D2D0","noteTextColor":"#3D3937","activationBkgColor":"#F1EAE7","activationBorderColor":"#D7D2D0","altSectionBkgColor":"#FAF8F6","altSectionColor":"#3D3937"},"fontFamily":"Inter, sans-serif"}}%%
flowchart TB
    A["Current Action:<br/>'Discuss test results'"] --> B{"Matches<br/>sensitive_topics?"}
    B -->|Yes| C["Preemptive shift to<br/>sympathetic tone"]
    B -->|No| D["Normal tone<br/>priority chain"]
```

`sensitive_topics` is configurable via [voice settings](https://docs.concurrence.com/developer-guide/platform-api/workspaces.md#voice-settings). Falls back to healthcare defaults: test results, diagnosis, billing, payment, insurance, denial, emergency, referral, specialist, surgery, procedure, medication.

This fallback applies only when navigation picks no tone for the reply. An explicitly configured workspace `tone` then overrides the result.

### Control Plane ↔ Adaptation

How each workspace voice setting interacts with automatic adaptation:

| Voice Setting                | What You Control                    | What the System Overrides                               | Override Condition                                                                           |
| ---------------------------- | ----------------------------------- | ------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| `tone`                       | Explicit workspace TTS tone         | Replaces sensitive-topic selection when configured      | Whenever the setting is present                                                              |
| `speed`                      | Base speech rate                    | Can be reduced by supported empathy pacing logic        | When an empathy baseline is active. Configured service floors apply                          |
| `volume`                     | Base volume                         | No adaptive override in the standard voice-context path | Provider support still applies                                                               |
| `voice_id`                   | Voice persona                       | Per-agent voice config overrides                        | Agent version has voice config set                                                           |
| `keyterms`                   | Domain vocabulary for STT boost     | Merged with service keyterms and deduplicated           | Subject to selected STT provider support                                                     |
| `sensitive_topics`           | Topics for proactive tone softening | Falls back to healthcare defaults                       | Preemptive, not reactive                                                                     |
| `post_call_analysis_enabled` | Retained workspace setting          | None                                                    | Does not establish an active automatic scoring pipeline. Use the documented evaluation paths |

**Key principle**: automatic tone selection is best-effort. Explicit workspace and service settings remain authoritative where the active runtime supports them.

### Graceful Degradation

The standard runtime implements the best-effort behaviors below. Optional post-call failures can leave those features unavailable without ending an otherwise healthy call, but failures in required telephony, speech, model, or context-engine dependencies can interrupt or end the session.

If speech recognition fails and can't reconnect on a platform-managed call, the agent leaves when an operator has taken over the call, and the operator keeps it. Otherwise the call switches once to another recognizer configured for the caller's language. If none is available or it can't connect, the call takes the safe exit: the agent plays the AI and recording notice if the caller hasn't heard all of it, then an apology, and cold-transfers the caller to the service's `fallback_to`, else its fixed `forward_to`. With neither, or if the platform can't hand the caller off, it plays a closing line that asks the caller to call back and, in an emergency, to hang up and call 9-1-1 or their local emergency number, then hangs up. Set `fallback_to` or a fixed `forward_to` on services that should hand these calls to a person.

A `gpt_realtime` call takes the same safe exit when something it needs fails before its realtime session starts: its agent, caller context, call control, tools, or workflow prompt can't be set up. It also takes it when its realtime session doesn't open: the connection to the realtime model fails or doesn't open in time, the call's session setup can't be sent, or the model doesn't answer the setup within a few seconds or refuses it, for example over a custom `realtime.voice` id it doesn't recognize. Nothing from the model plays. The caller hears the notice if they haven't heard all of it, then the apology and a transfer, or the closing line and the end of the call. A call whose service configuration can't be loaded when it arrives, after retries, has no runtime to run on and takes the safe exit whichever runtime the service uses.

A platform-managed call takes the same safe exit when its agent can't be set up, for example because the agent's configuration doesn't finish loading in time. On either runtime, a call whose setup fails after its service configuration has loaded plays the exit's lines in the language set in the agent's identity, not the caller's record language or the workspace voice language: Spanish for an agent whose default language is Spanish, English then Spanish for a multilingual agent, and English for any other language. When the service has a `fallback_to` or a fixed `forward_to`, the exit also cold-transfers the caller to the `fallback_to`, else to the `forward_to`. A call with no runtime to run on has no agent language, `fallback_to` or `forward_to` to use, so its exit plays in English and isn't handed to either. On an outbound call, on either runtime, a setup that fails before the callee answers ends the dial before anyone answers. In the Playground, a test call whose service configuration can't be loaded, or whose agent can't be loaded for a reason the platform can name, is refused instead, with the error. On the platform-managed runtime any other agent-load failure is refused too; on `gpt_realtime`, any other setup failure plays the safe exit in the browser, which has no phone line to transfer, so it ends with the closing line. See [Test Call Error Reporting](https://docs.concurrence.com/testing/testing/playground#test-call-error-reporting).

On both runtimes, a caller's turn that gets no reply takes the safe exit too. On the platform-managed runtime, that covers a model that stalls, speech output that stops, and a platform dependency that never answers. On `gpt_realtime`, it covers a turn the model never answers. A `gpt_realtime` session that ends for good during a call ends the call rather than leaving the caller on a silent line, unless a transfer has already handed the caller to someone.

If the agent can't join an inbound call, the caller hears the closing line, in English, and the call ends.

Every line the agent plays in the safe exit is fixed audio prepared ahead of time, in one voice, so playing it doesn't depend on speech synthesis during the call. A `gpt_realtime` call always plays that audio, so neither `realtime.voice` nor the agent's text-to-speech voice applies to these lines. A platform-managed call plays them in its own text-to-speech voice when it has prepared them in the language the call speaks, and the prepared audio otherwise, as on a call whose agent can't be set up. The closing line heard by a caller the agent can't join is read by the telephony provider in its own voice.

On a phone call, the safe exit dials the service's `fallback_to` when it has one, else its fixed `forward_to`. With neither, it plays the closing line and ends the call. The transfer is cold even when `warm_transfer` is on, and the agent doesn't choose the number. Channel Manager applies the use case's `allowed_forwarding_numbers` to this transfer as it does to a transfer the agent starts, and saving the service doesn't check that list, so confirm the number is on it before you set `fallback_to` on a live service. A number that isn't on it is refused when the exit dials it, and the exit then plays the closing line and ends the call. When the exit has handed the caller to the number and that line is busy, doesn't answer, or fails, the call ends. See [Forwarding destinations](https://docs.concurrence.com/developer-guide/platform-api/conversations/channel-manager.md#control-voice-calls).

In the Developer Console, open the service under **Build > Agents** and choose **Edit**. In the **Call forwarding** section, set **Mode** to Static or Dynamic, because with Disabled there is no forwarding block and so no fallback number. Enter the number in the field under **If the agent can't continue** and choose **Save changes**. Clear the field to remove it.

<figure><img src="https://1651797362-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YwM0LHtEGLt9S21OyC3%2Fuploads%2Fgit-blob-fb933a42fc6c1f7c94507bfeba1fd6a9f33e93bd%2Fstt-failure-safe-exit.svg?alt=media" alt="When speech recognition fails and can&#x27;t reconnect on a platform-managed call, the agent leaves if an operator has taken over the call, or the call switches once to another recognizer. If that isn&#x27;t possible, the agent plays the AI and recording notice if the caller hasn&#x27;t heard all of it, then apologizes and cold-transfers to a fixed forward_to, or plays a closing line and hangs up."><figcaption><p>If the caller hasn't heard all of the AI and recording notice, it plays before the apology. The notice, the apology, and the closing line play in English or Spanish, and other languages hear English. On <code>gpt_realtime</code>, and on a platform-managed call whose setup fails, the caller of an agent whose default language is multilingual hears each line in English, then Spanish. The figure shows a service with a fixed <code>forward_to</code>. A service with a <code>fallback_to</code> transfers the caller to that number instead, and a dynamic service without one takes the closing line.</p></figcaption></figure>

| Layer                    | Failure Mode                                | Fallback                                                                                                                                                                                        | Impact                                                                                                          |
| ------------------------ | ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| **Voice settings**       | Parse error                                 | Defaults (filler on)                                                                                                                                                                            | Baseline experience still works                                                                                 |
| **Post-call analysis**   | Any error                                   | Logged, not raised (fire-and-forget)                                                                                                                                                            | Quality data missing, call unaffected                                                                           |
| **TTS connection**       | Close/error mid-stream                      | A later turn can attempt reconnection                                                                                                                                                           | Audio gap. A caller's turn left with no reply takes the safe exit, which doesn't need speech synthesis          |
| **STT connection**       | Connection loss                             | Reconnection is attempted. After a lasting failure, the path in the figure above                                                                                                                | Transcription gap while recovering. The call can transfer or end                                                |
| **Reply generation**     | Model error, or a reply that stops arriving | A request that fails with a transient error before the reply starts can be retried once while time allows. Otherwise the reply ends, and a caller's turn that gets no reply takes the safe exit | What remains of that reply does not play                                                                        |
| **Context graph engine** | Backend unavailable                         | Any configured runtime fallback is attempted                                                                                                                                                    | The response can fail when no fallback is available, and a call whose agent can't be set up takes the safe exit |

## Tool Execution

Tool bindings can execute inline or in the background during calls. Background bindings let the agent acknowledge the action and continue speaking while results arrive through the configured continuation-delivery mode.

<figure><img src="https://1651797362-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YwM0LHtEGLt9S21OyC3%2Fuploads%2Fgit-blob-c5ceeba7d4273bec6bd1e96adbf14e4040dc0eb3%2Fvoice-tool-execution.svg?alt=media" alt="An eligible tool can execute inline within a reasoning transaction or in the background while conversation continues. Background result lifecycle and delivery are configured separately from voice playback."><figcaption><p>Filler is optional. A tool result does not guarantee a spoken reply or a successful external mutation.</p></figcaption></figure>

On the platform-managed runtime, one caller turn can run up to six rounds of tool calls. After the sixth, the agent answers with the results it has, and the count starts over when the caller speaks. If one request needs a longer chain, combine steps into fewer tools or a companion skill.

<figure><img src="https://1651797362-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YwM0LHtEGLt9S21OyC3%2Fuploads%2Fgit-blob-6b9b93d10e29bbcefaf2d8a1b358b27efabce677%2Ftool-rounds-per-turn.svg?alt=media" alt="On the platform-managed runtime, one caller turn can run up to six rounds of tool calls, each returning its results together. After the sixth, the agent answers with the results it has, and the count starts over when the caller speaks."><figcaption><p>The results of one round's tool calls return to the agent together.</p></figcaption></figure>

### Code-Entry Tools <a href="#code-entry-tools" id="code-entry-tools"></a>

Set `expects_code` to `true` on a state's tool call spec (default `false`) for a tool that takes a spoken code, such as one the caller reads from a text message. While that tool is the next step, voice turn detection waits longer before it ends the caller's turn, so the caller can read the code in groups without the agent replying between them. The setting changes only how voice turns end. The tool, its parameters, and its result are unchanged.

### Execution Model

Tool calls are routed by tool family, and each state-level tool binding declares how the call executes and how its result is delivered:

| Tool Family                | Prefix      | Execution Model                                                                                                                                                                                                 |
| -------------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **World tools**            | `world_*`   | Built-in reads/writes against the world model (patient, appointment, insurance, typed entity lookups)                                                                                                           |
| **Surface tools**          | `surface_*` | Create, deliver, and track [surfaces](https://docs.concurrence.com/developer-guide/platform-api/conversations/surfaces.md)                                                                                      |
| **Skills**                 | (by slug)   | LLM-backed companion micro-agents: multi-turn reasoning with the skill's configured integration and static tools (see [Skills](https://docs.concurrence.com/developer-guide/platform-api/workspaces/skills.md)) |
| **Platform functions**     | `fn_*`      | Registered [platform functions](https://docs.concurrence.com/developer-guide/platform-api/functions.md) executed against the data warehouse                                                                     |
| **Workspace data queries** | `wsq_*`     | Parameterized SQL against [workspace tables](https://docs.concurrence.com/developer-guide/platform-api/functions/workspace-data-queries.md)                                                                     |
| **System tools**           | -           | `forward_call`, `complete_transfer`, `resume_patient`, `get_tool_call_status`, `cancel_background_task`, handled inline by the turn controller                                                                  |

**Blocking vs background execution**: each state x tool binding declares an `execution` axis. Blocking tools are awaited inline. Background tools start without being awaited, so the agent can acknowledge and keep talking. When background execution and delivery succeed, the result can arrive as a continuation turn. The binding's `delivery` axis controls how that result lands: `interrupt` (a continuation turn that can cut current speech) or `queue` (folded in on the next user turn). A separate `failure_delivery` axis can route reported failures differently from successes.

**State gating**: in the standard HSM runtime, tool calls are validated against the tools offered for the current context-graph state, and an unavailable call is rejected. Native and realtime runtimes can also attach shared platform or system tools globally to each phase. In every runtime, authorization, explicit tool enablement, and patient/write scope remain the security boundary. State membership alone is not one.

**In-flight write deduplication**: an identical write call is suppressed only while the first call is still running. It does not deduplicate a later retry after completion and does not provide request-level exactly-once semantics.

**Writes with an unknown outcome**: an integration write that got no answer saying whether it went through reports `"outcome": "unknown"`. On both runtimes, the platform doesn't send it again in that conversation, and the agent is told to say it couldn't confirm the request went through. Reconcile these writes in the destination system. See [Endpoint Retries](https://docs.concurrence.com/developer-guide/platform-api/integrations.md#endpoint-retries).

**Repeated requests**: when the caller asks again for a lookup or action already done in the conversation, the agent is prompted to call the tool again rather than repeat the earlier result. This is model behavior, so test repeated requests in your flows.

**Approval-gated tools**: voice calls on either runtime don't offer gated endpoints or skills marked `approval_required`, so move those steps into a text flow or remove the gate. See [Approval-Gated Integration Writes](https://docs.concurrence.com/developer-guide/platform-api/integrations/approval-gated-integration-writes.md#key-concepts).

### Built-in Clinical Tools

The Platform API publishes a catalog of built-in `world_*` tools. A supported runtime can expose eligible catalog entries after applying service configuration, Context Graph bindings, authorization, and entity scope. Catalog presence is therefore discovery information, not proof that a tool is callable in every session.

**Read tools:**

| Tool                                                                                                                                                                                                                                                                                                                                                                                                                            | Purpose                                                                                                                                                            |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `world_patient_lookup`                                                                                                                                                                                                                                                                                                                                                                                                          | Search patients by DOB, name, phone, or MRN (DOB preferred for accuracy)                                                                                           |
| `world_confirm_patient_identity`                                                                                                                                                                                                                                                                                                                                                                                                | Verify the caller: compares the stated full name and date of birth with the session's candidate records and loads the record when exactly one matches              |
| `world_persons_lookup`                                                                                                                                                                                                                                                                                                                                                                                                          | Generic person-entity lookup with server-side filters                                                                                                              |
| `world_slot_search`                                                                                                                                                                                                                                                                                                                                                                                                             | Available appointment slots by location and date. Returns human-readable times and doctor names while retaining the identifiers needed for a later scheduling call |
| `world_appointment_lookup`                                                                                                                                                                                                                                                                                                                                                                                                      | Patient's existing appointments, with references for cancel/confirm/reschedule                                                                                     |
| `world_payer_search`                                                                                                                                                                                                                                                                                                                                                                                                            | Look up insurance payers/carriers                                                                                                                                  |
| `world_reference_data_lookup`                                                                                                                                                                                                                                                                                                                                                                                                   | Look up reference data (locations, appointment types, providers)                                                                                                   |
| `world_get_medications`                                                                                                                                                                                                                                                                                                                                                                                                         | Patient's medications and prescription references                                                                                                                  |
| `world_get_encounters`                                                                                                                                                                                                                                                                                                                                                                                                          | Patient's encounter history                                                                                                                                        |
| `world_queue_lookup`                                                                                                                                                                                                                                                                                                                                                                                                            | Read the queue of pending outbound call tasks                                                                                                                      |
| `world_conditions_lookup`, `world_allergies_lookup`, `world_observations_lookup`, `world_medication_statements_lookup`, `world_clinical_notes_lookup`, `world_family_history_lookup`, `world_questionnaire_responses_lookup`, `world_intake_uploads_lookup`, `world_care_plans_lookup`, `world_goals_lookup`, `world_tasks_lookup`, `world_service_requests_lookup`, `world_consents_lookup`, `world_diagnostic_reports_lookup` | Typed read tools over projected entity shapes                                                                                                                      |
| `world_conversations_lookup`, `world_calls_lookup`                                                                                                                                                                                                                                                                                                                                                                              | Conversation and call record lookups                                                                                                                               |
| `world_state_resolve`                                                                                                                                                                                                                                                                                                                                                                                                           | Batch entity-ID to current world-state resolver                                                                                                                    |

**Write tools:**

| Tool                                 | Purpose                                                                                                     |
| ------------------------------------ | ----------------------------------------------------------------------------------------------------------- |
| `world_patient_create`               | Create a patient after the supported duplicate checks; ambiguous matches still require reconciliation       |
| `world_patient_update`               | Update contact info (phone, email, address); requires an entity reference                                   |
| `world_save_patient`                 | Create or update after the supported duplicate check; accepts natural field names and flexible date formats |
| `world_schedule_appointment`         | Book from a supported slot-search result and resolve its scheduling identifiers                             |
| `world_cancel_appointment`           | Cancel by appointment reference                                                                             |
| `world_confirm_appointment`          | Confirm a booked appointment                                                                                |
| `world_reschedule_appointment`       | Move an appointment to a new slot                                                                           |
| `world_insurance_create`             | Record an insurance event for the patient                                                                   |
| `world_refill_prescription`          | Submit a refill request for a specific prescription                                                         |
| `world_log_call`, `world_log_triage` | Record call and triage outcomes                                                                             |
| `world_ticket_create`                | Create a support-ticket event for a supported downstream workflow                                           |

World write results carry source and confidence metadata. A world write is not delivered to an external system; an external change needs its own integration call. Entity-anchored tools also apply the available [write scope](#patient-safety-isolation); catalog membership alone is not a delivery or safety guarantee.

### Call Forwarding

The service's `voice_config.forwarding` controls whether the runtime exposes `forward_call`. A null or omitted block disables forwarding. When configured, `forward_to` selects a fixed E.164 destination. When it is null, the service uses dynamic mode, and `forward_call` offers only the numbers in `approved_destinations`, each with its label. Any other number is refused without dialing, and a dynamic service with no approved destinations offers no `forward_call`. `warm_transfer` must explicitly select warm or cold transfer. There is no workspace destination fallback.

`fallback_to` is not a mode. It is the line the safe exit dials in either mode, and `forward_call` never offers it, though a number the service also lists in `approved_destinations` is offered as an approved destination. It is a service setting, optional in both modes. A dynamic service that approves no destinations offers no `forward_call` and can still name one. `forward_call` doesn't use it: with a fixed `forward_to`, `forward_call` still dials that number, and in dynamic mode a number listed only as `fallback_to` is refused as unapproved. Only the safe exit dials it. See [Graceful Degradation](#graceful-degradation). The request format, an example, and the refusals are in [Voice Configuration](https://docs.concurrence.com/developer-guide/platform-api/workspaces/voice-configuration.md#call-forwarding).

Confirm destination approval in the provisioned channel as well, `fallback_to` included. Channel Manager enforces a per-use-case forwarding allowlist, `allowed_forwarding_numbers`, separately from service configuration. See [Forwarding destinations](https://docs.concurrence.com/developer-guide/platform-api/conversations/channel-manager.md#control-voice-calls).

<figure><img src="https://1651797362-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YwM0LHtEGLt9S21OyC3%2Fuploads%2Fgit-blob-300ba91c4b951afc69db378d1ab03ddc00d94b3e%2Ftransfer-resolution.svg?alt=media" alt="The forwarding block offers forward_call to a fixed number or, in dynamic mode, only to approved destinations, and each must be on the line&#x27;s allowlist. A transfer that can&#x27;t connect, including a warm transfer that misses, fails, or loses the person who answered before the hand-off, returns the caller to the agent."><figcaption><p>The service's forwarding block chooses the destination, and the line's forwarding allowlist applies on top of it.</p></figcaption></figure>

On the platform-managed runtime, state policy can further restrict forwarding. Authored instructions to transfer do not prove that the tool was invoked or that the destination answered.

On the conference warm-transfer path, the caller stays on hold while the destination is dialed. The briefing waits for a human-answer classification. Once a person answers, the caller comes off hold and hears the agent's briefing, and caller speech doesn't interrupt it. On both runtimes, the [reply checks](https://docs.concurrence.com/operations-and-safety/runtime-safety#reply-checks) review the briefing as they do any other reply. Voicemail, a fax, no answer, a hold or dial that fails, or a person who answers and hangs up before the hand-off takes the caller off hold and returns them to the agent, which is told to let them know and keep helping. Until a person answers, what the agent hears on the dialed line is not treated as the caller speaking. This applies on both runtimes. On `gpt_realtime`, a session that ends while the agent is briefing the person hands the caller to them. Before the agent leaves, the platform confirms the caller is off hold, and if it can't, the agent stays on the call. Provider answer classification is fallible.

While a transfer is under way, through its hold, dial, and briefing, the agent can't end the call, on either runtime: a goodbye said with the transfer plays, and the transfer goes ahead. Once a transfer hands the caller back, the agent can end the call again. After a completed cold transfer, ending the call closes only the agent's leg.

If the platform cannot hand the caller off on a cold transfer, for example because the line's forwarding allowlist does not include the destination, the caller stays on the call with the agent and `forward_call` resolves as failed. The platform-managed runtime gives the agent a follow-up turn with the failure so it can tell the caller and keep helping. On `gpt_realtime` the model reads the failure as the tool result and is instructed to tell the caller and keep helping. Once the hand-off succeeds, the caller has left the agent's call.

See [Voice Configuration](https://docs.concurrence.com/developer-guide/platform-api/workspaces/voice-configuration.md) for authoring and [Operators and Escalation](https://docs.concurrence.com/operations-and-safety/operators) for the operational sequence.

## Safety & Escalation

Safety behavior comes from the agent definition, Context Graph guidance, the state's turn policy on the platform-managed runtime, eligible tools, and operator workflows. Model reasoning can still miss or misclassify a concern, so these controls require scenario testing and a human fallback. Supported escalation paths include:

* **Call forwarding**: the agent invokes the built-in [`forward_call` tool](#call-forwarding) to transfer the caller to a human.
* **Operator join**: a human operator joins in listen or takeover mode. Takeover suspends agent speech while the operator drives, and listen mode does not.
* **Silence**: on both runtimes, quiet after the agent speaks brings up to two check-ins, and the next silence ends the call, unless the caller has described an emergency. See [Terminal State & Auto-Hangup](#terminal-state-and-auto-hangup).
* **Emergencies**: once the caller describes a medical emergency or self-harm, the agent doesn't end the call on either runtime, unless a `gpt_realtime` service is set to end it after the script for a medical emergency, and on `gpt_realtime` it speaks an authored emergency script first. See [Terminal State & Auto-Hangup](#terminal-state-and-auto-hangup) and [Realtime Emergency Script](#realtime-emergency-script).
* **Speech recognition failure**: a platform-managed call switches once to another recognizer when one is available. If that isn't possible, the call takes the safe exit: the AI and recording notice if the caller hasn't heard all of it, an apology, and a cold transfer to the service's `fallback_to`, else its fixed `forward_to`, or a closing line and a hangup. See [Graceful Degradation](#graceful-degradation).
* **Call setup**: a platform-managed call takes the same safe exit when its agent can't be set up, and a `gpt_realtime` call when anything it needs fails before its realtime session starts, or when its realtime session doesn't open, because the connection fails or times out, the session setup can't be sent, or the realtime model refuses or doesn't answer it. Once the service configuration has loaded, that exit plays in Spanish for an agent whose default language is Spanish, in English then Spanish for a multilingual agent, and in English otherwise, and, when the service has a `fallback_to` or a fixed `forward_to`, cold-transfers to the `fallback_to`, else to the `forward_to`. A call whose service configuration can't be loaded when it arrives plays the exit in English and isn't handed to either. See [Graceful Degradation](#graceful-degradation).
* **No reply**: on both runtimes, a caller's turn that the agent doesn't answer takes the same safe exit. See [Graceful Degradation](#graceful-degradation).
* **Request for a person**: a turn recognized as a request for a person does not end the conversation, and on `gpt_realtime` the agent is instructed the same way. The paths it is told to take are below.

When someone asks for a person, the agent is told to take the first path that fits:

1. **Transfer** with `forward_call` when the call can transfer.
2. **Take a message**, only when its current action state offers a tool whose write records one: `world_ticket_create`, an integration endpoint [declared `message`](https://docs.concurrence.com/developer-guide/platform-api/integrations.md#endpoint-write-declarations), or a companion skill that can call one of those endpoints without an approval gate (a `varies` endpoint counts when the skill's `write_kind` is `message`). It is told to record what the caller needs and how to reach them, and to confirm the message only after the write succeeds.
3. **Explain how to reach the team**, from a number in its instructions or a tool result, or where to find one, such as the website or a recent bill, letter, or appointment reminder, with office hours when known.

<figure><img src="https://1651797362-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2YwM0LHtEGLt9S21OyC3%2Fuploads%2Fgit-blob-e574c6cc0ca29678db315bb189c59bd30a4e9f03%2Fperson-request-paths.svg?alt=media" alt="When someone asks for a person, the agent is told to transfer if it can, otherwise to take a message through a tool that records one, and otherwise to explain how to reach the team, without making up a number or promising a callback."><figcaption><p>Voice and text follow the same rule. On <code>gpt_realtime</code>, the agent is told to transfer when the call has <code>forward_call</code> and otherwise to explain how to reach the team, without offering a message.</p></figcaption></figure>

When it explains how to reach the team, the agent is told never to make up a number or promise a callback. If an earlier message through the message tool has an [unknown outcome](#execution-model), it is told to say it couldn't confirm the message went through instead of recording another. The voice reply checks stop a message claimed as taken when no message write succeeded. The [text reply check](https://docs.concurrence.com/developer-guide/platform-api/conversations.md#reply-check) judges the same claim within the turn and only logs it. On `gpt_realtime` without `forward_call`, the agent is told that nothing on the call can take a message.

The crisis-resource guidance, the reply checks on voice and text, the self-harm script on text, and the emergency script on `gpt_realtime` are described in [Runtime Safety](https://docs.concurrence.com/operations-and-safety/runtime-safety#control-map).

The Console does not provide takeover navigation or operator pages. Use All runs and its Live status filter to inspect available active records. Intervention requires a separate integration with the operator APIs and a supported runtime. See [Operators](https://docs.concurrence.com/developer-guide/platform-api/conversations/operators.md) for those resources and [Safety & Monitoring](https://docs.concurrence.com/developer-guide/platform-api/safety.md) for the safety model, audit trail, and compliance reporting.

### Realtime Crisis Line <a href="#realtime-crisis-line" id="realtime-crisis-line"></a>

Every `gpt_realtime` session opens its instructions with the platform's crisis rule and the deployment region's crisis resource. The rule covers hints as well as plain words, for example saying they won't be around after a certain day, that they've said their goodbyes, or that others would be better off without them. In its reply to the first such sign, the agent is told to give the resource.

This is model guidance, not a deterministic check. Test realtime services with plain and hinted cases.

### Realtime Emergency Script <a href="#realtime-emergency-script" id="realtime-emergency-script"></a>

On every `gpt_realtime` call, a caller line that the platform recognizes as describing a medical emergency or self-harm, in English or Spanish, cuts off any reply the agent is speaking, and the agent speaks an authored script, in English or Spanish to match the words it recognized, before anything else. Outside the call opening, caller speech cuts the script off, as it cuts any reply. For a medical emergency, the script tells the caller to hang up and call the deployment region's emergency number now. For self-harm, it gives the region's crisis line and the emergency number for immediate danger, or, where the region has no crisis line, asks the caller to call the emergency number now, and it tells the caller the agent is still with them. A line that describes a medical emergency together with self-harm gets the medical emergency script. On a call in the US region, the same script then gives the crisis line, in English or Spanish to match the caller. During a warm transfer's briefing, recognition doesn't cut the briefing off, and the script isn't queued after it. During the call opening, a notice the script cuts off plays again in full after it, and the greeting is dropped.

The realtime model speaks the script from the authored text, in the session's voice, so its wording can vary. Recognition covers a limited set of phrasings and can miss others, and it can play the script when a caller only mentions a condition, so test realtime services with plain, hinted, and non-emergency cases. When recognition misses suicidal thoughts, self-harm, or an overdose, the crisis rule above still guides the agent's reply. Any other emergency it misses is answered from the agent's own instructions, so cover emergencies in the action state's guidelines too. On the platform-managed runtime, the agent's own reply plays instead, so author the graph's emergency path for it.

After the script, the call stays open. See [Terminal State & Auto-Hangup](#terminal-state-and-auto-hangup). A `gpt_realtime` service with `medical_emergency_ending: end_after_script` ends the call instead after a medical emergency described in English: the script is followed by "I'm going to end this call now so you can call 9-1-1.", with the region's number, and the call ends once both have played in full, as written. Self-harm, or an emergency described in Spanish, keeps the call open for the rest of it whatever the setting. A caller who talks over the two lines, or a transfer that is under way, also keeps the call open.

## Observer WebSocket

Monitor active calls in real time via a WebSocket connection:

```
WS /agent/observe/{call_sid}
Sec-WebSocket-Protocol: auth, {api_key}
```

Requires a valid workspace API key or an identity token, sent only in the `Sec-WebSocket-Protocol` header as `auth, <token>`. The server accepts the `auth` subprotocol. A `?token=` query parameter is not read, and a connection without the subprotocol closes with `4001`. The Platform SDK's `observers.subscribe` sends the credential this way. Any observer can monitor any active call in the workspace, regardless of which server handles the call.

**Late-join replay**: observers connecting mid-call receive a buffered replay of recent events before transitioning to the live stream. Events carry monotonic sequence numbers for ordering.

**In the Developer Console**: open a running voice run under **Evaluate > Runs** and click **Observe live call** to watch the call and send guidance. The link appears only while a voice run that is not a simulation is running.

**Reconnecting**: the server can close an observer connection with code `1012` when it restarts. Reconnect to keep watching the call. The Platform SDK's `observers.subscribe` reconnects on its own.

### Observer Event Types

The observer stream emits session lifecycle, transcript, state transition, tool, timing, empathy, and call control events. The full catalog with field lists and PHI annotations lives on the observer events page:

{% content-ref url="/pages/Qm4yWkfviMbMOVLEbCzJ" %}
[Observer Events](https://docs.concurrence.com/developer-guide/platform-api/conversations/observer-events.md)
{% endcontent-ref %}

## Session Event Injection

External systems can inject events into supported platform-managed voice sessions. The separate realtime-provider runtime does not process these injections. When delivery and response generation succeed, the platform-managed runtime handles the event without Context Graph navigation and can speak a response. Injection acceptance does not guarantee response timing, exact wording, or that a response will be produced.

### Injection Paths

Use the Platform session injection operation below. The `/test-call` WebSocket also accepts control frames for developer playground testing.

#### Platform Session Injection

Use workspace authentication for frontend or third-party injection.

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/sessions/{call\_sid}/inject" method="post" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

#### Operator Guidance

Use operator-scoped guidance when the caller should be identified and checked for `Operator:Update` permission.

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/operators/{operator\_id}/send-guidance" method="post" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

### Injected Event Types

| Type             | Behavior                      | Example                                        |
| ---------------- | ----------------------------- | ---------------------------------------------- |
| `external_event` | Queues behind current speech. | "Appointment confirmed for 2pm tomorrow"       |
| `guidance`       | Interrupts current speech.    | "Ask for their insurance ID before confirming" |

The agent's response to either counts as agent speech for the silence rules in [Terminal State & Auto-Hangup](#terminal-state-and-auto-hangup): the next 10 seconds of quiet count from when it ends, and an injected event doesn't start the check-in count over.

The distinction matters: external events carry factual information that can wait for the agent to finish speaking, while guidance is prioritized as an instruction and can interrupt current speech. Delivery can still be delayed or fail, and the model is not guaranteed to follow the instruction verbatim.

### Response

The endpoint returns delivery status indicating whether the event was received by the session:

```json
{
  "status": "delivered",
  "call_sid": "2c725f98-a9c6-4be2-8a48-6eb42d1f81cb"
}
```

A `status` of `"queued_no_subscriber"` indicates the event was published but no active session was listening. This can happen during a brief window when a session is initializing or if the call has already ended. A `status` of `"deduplicated"` indicates that an identical injection (same message, sender, and event type) for the same call was suppressed within a 30-second window. This reduces duplicate processing for ordinary retries but is not an end-to-end delivery or exactly-once guarantee.

### Delivery Resilience

Injection can cross runtime instances to reach the active call. Inspect the returned status: `delivered` confirms that a session listener received the publication, not that the model followed it or that a caller heard a response. Runtime interruptions can delay or prevent later processing.

### Active Sessions

List currently active sessions via the platform API:

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/sessions/active" method="get" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

Returns a best-effort real-time list of active sessions with call metadata. A live session can appear late or be temporarily absent, so use the canonical Runs surface for durable inventory.

A trusted backend can use an entry's `call_sid` to set verified context on that call. Match exactly one entry before doing so. See [Set Context on a Live Voice Call](https://docs.concurrence.com/developer-guide/guides/live-call-context.md#find-the-call).

### WebSocket Control Channel

Test calls and direct streams accept text-frame control messages for injection and session control:

```jsonc
// Inject an external event
{"type": "inject_event", "message": "...", "sender": "..."}

// Inject operator guidance (interrupts speech)
{"type": "inject_guidance", "message": "..."}

// Force context refresh (reloads patient data)
{"type": "refresh_context"}

// Stop the session
{"type": "stop"}
```

### Test-Call Scenarios

The `/test-call` WebSocket endpoint supports scenario-based testing:

| Parameter                 | Default      | Description                                                                                                 |
| ------------------------- | ------------ | ----------------------------------------------------------------------------------------------------------- |
| `scenario`                | `inbound`    | `inbound` (agent greets first), `outbound` (task context greeting), `silent` (no greeting)                  |
| `caller_id`               | empty string | Simulated caller phone number. Test calls are tagged with source `playground` independently of `caller_id`. |
| `outbound_task_entity_id` | -            | Entity ID for outbound task context (required for `outbound` scenario)                                      |
| `system_prompt`           | -            | Freeform prompt override (takes precedence over scenario-derived prompts)                                   |

```
WS /agent/test-call?encoding=mulaw&scenario=outbound&outbound_task_entity_id=123&caller_id=+15551234567
Sec-WebSocket-Protocol: auth, {api_key}
```

Send the credential in `Sec-WebSocket-Protocol`, as on the observer socket. A `?token=` query parameter is not read, and a connection without the `auth` subprotocol closes with `4001`. The audio encoding must be `mulaw`, and any other `encoding` closes with `4008`. A workspace can hold up to 5 test calls at once, and starting another returns `429` until one ends.

A test call always runs in the credential's workspace. A `workspace_id` query parameter that names a different workspace closes the socket with code `4003` before the session starts. Omitting the parameter uses the credential's workspace.

## Call Record & Persistence

Realtime-provider phone and playground calls attempt to open a conversation record when the call starts and finalize the same record when it ends. Finalization records the end time, duration, completion reason, final state, and a turn count that accounts for provider-reported turns and captured transcript/tool entries. Call startup can continue after a persistence failure, and failed finalization can leave a record non-terminal.

For `gpt_realtime`, completed user and agent transcripts and completed tool calls are retained in the provider's conversation-item order, rather than the order in which asynchronous transcription finishes. Tool entries include the name, call identifier, arguments, full output, duration, and success/failure. An observer's shortened tool-output preview is not the full retained result.

History is written incrementally during the call. An unfinished earlier item can delay later entries. At teardown, unfinished items are skipped and completed unsaved entries are retried. Missing provider completions or failed writes can still leave gaps. A retained lifecycle record, turn count, or live observer event does not establish complete history, recordings, or quality analysis. This transcript and tool behavior is specific to the `gpt_realtime` runtime.

The platform-managed pipeline normally persists the call record at completion and can also retain intermediate checkpoints. Use [Runs](https://docs.concurrence.com/developer-guide/platform-api/conversations/runs.md) for the relationship between live visibility and retained history. Depending on the runtime, enabled capture, and successful persistence, the record can include:

* **Turns**: each turn carries a timing model with four field groups (all fields in milliseconds):
  * **User speech boundaries**: `user_speech_start_ms`, `user_speech_end_ms` (when the caller's speech started and ended).
  * **Processing breakdown**: `engine_ms`, `nav_ms`, `render_ms`, `audio_ttfb_ms` (reasoning-engine, navigation, render, and audio time-to-first-byte latency).
  * **Agent speech boundaries**: `agent_speech_start_ms`, `agent_speech_end_ms` (when the line's first and last agent audio were sent, which is ahead of when the caller heard them). Each line that played has its own, a filler and the reply that follows it included.
  * **Audio window**: `audio_window_start_ms`, `audio_window_end_ms` (the turn's bounding audio interval).
* **Tool calls**: name, input, output, duration, success/failure.
* **State transitions**: recorded context graph navigation events.
* **Escalation history**: recorded escalation lifecycle events when an operator joined.

For what ran the call, read `session_provider` and `manifest` on the [run](https://docs.concurrence.com/developer-guide/platform-api/conversations/runs.md#what-ran-a-run) or conversation detail.

## Calls API

### Voice Run Inventory

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/runs" method="get" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

`/runs` is the canonical workspace inventory for current clients. Filter by live status, conversation kind, and voice channel for active calls, and omit the status filter for history. Active-call visibility is best-effort, so clients should tolerate a live call appearing late or being temporarily absent.

### Compatibility Voice Listing

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/calls" method="get" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

This voice-specific compatibility listing is deprecated and returns `Deprecation: true` and `Sunset: 2026-07-01` response headers. Migrate inventory reads to `/runs`.

### Compatibility Call Detail

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/calls/{call\_id}" method="get" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

Available call-record fields can include turns with timing data, tool calls, state transitions, escalation history, and safety state. Turns are listed in the order they sit in the recording, and each timeline turn's `turn_index` is its row in `turns`. To review a saved platform-managed call however long ago it ended, read [conversation detail](https://docs.concurrence.com/developer-guide/platform-api/conversations.md#voice-call-detail), whose turns carry their place in the recording.

This compatibility detail route returns the same deprecation and sunset headers. Use `/runs/{run_id}` for the canonical run record. Call-specific subordinate analytics and recording routes remain documented below where no run-scoped replacement is available.

### Recordings

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/recordings/{call\_sid}" method="get" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

The operation checks workspace access and issues temporary links to stored caller, agent, and stereo audio. It does not check whether the objects exist before issuing the links. A successful response is therefore not proof that recording finalization completed. Handle missing objects and expired links when downloading.

Recordings store separate 8 kHz mu-law mono WAV files and create a decoded PCM stereo WAV during finalization, on platform-managed and `gpt_realtime` calls alike. Read each file's header for its codec and sample rate. Some calls lack the merged stereo artifact even when mono files exist.

There are no separate metadata, URL-suffix, or download-proxy endpoints. Playback clients fetch the links and derive metadata or waveforms from the audio. See [Call Recordings](https://docs.concurrence.com/channels/voice/recordings) for retention and availability boundaries.

### Outbound Calls

Outbound calls are placed through the Platform API outbound call endpoint, which handles use-case-based caller ID selection, an explicit startup instruction on the `concurrence` runtime, response reuse for some same-key retries, and best-effort world-model lineage. The patient identifier is optional; when given, it validates and anchors lineage but is not passed as a dedicated voice-runtime context field:

{% content-ref url="/pages/B2fvwH4YLDTlmhMXUy84" %}
[Calls](https://docs.concurrence.com/developer-guide/platform-api/conversations/calls.md)
{% endcontent-ref %}

## Text Sessions

Text conversations - REST, WebSocket, SMS, and WhatsApp - can use the same Context Graph definitions and many of the same tool families as voice calls, but transport behavior, runtime tool exposure, turn handling, safety fallbacks, and persistence details differ by channel. The canonical documentation for text transports lives on [Conversations](https://docs.concurrence.com/developer-guide/platform-api/conversations.md): WebSocket connection and close codes, the `tool_events` default, SSE streaming event shapes, message queuing and coalescing, outbound SMS parameters, and inbound SMS and WhatsApp behavior.

{% content-ref url="/pages/js8kvebm1nrcYWgMPlI0" %}
[Conversations](https://docs.concurrence.com/developer-guide/platform-api/conversations.md)
{% endcontent-ref %}

### WhatsApp Voice Notes

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/services/{service\_id}/voice-turn" method="post" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

This operation transcribes a recorded audio turn, runs the configured reasoning path, and returns synthesized speech to the calling integration. The integration is responsible for delivering that audio to WhatsApp or another destination. The operation does not send a managed WhatsApp reply.

The response defaults to OGG Opus. A client that prefers WAV through content negotiation can receive PCM WAV instead. The embedded contract defines the accepted request, response media types, silence response, and concurrent-turn conflict.

See [Process a Voice-Note Turn](https://docs.concurrence.com/developer-guide/platform-api/workspaces/services.md#process-a-voice-note-turn) for how the platform picks a note's language and for the errors to handle.

Voice-note sessions use workspace, service, and phone number for continuity. Their state remains separate from managed WhatsApp conversations, which use business-scoped recipient identity and asynchronous text delivery. Voice notes do not provide live barge-in or operator takeover.

## Agent Trace & Debugging

Three endpoints provide deep inspection into agent reasoning and call quality.

### Execution Trace

```
GET /agent/calls/{call_id}/trace
```

Per-turn execution log showing recorded agent decisions. Each turn can include:

| Field           | Type           | Description                                                                            |
| --------------- | -------------- | -------------------------------------------------------------------------------------- |
| `action`        | string         | The action taken (speak, navigate, tool call, escalate)                                |
| `signal_kind`   | string         | What triggered this turn (caller speech, tool result, state transition, barge-in)      |
| `effect_kind`   | string         | What the agent did (spoke, invoked tool, navigated, escalated)                         |
| `tools`         | array          | Tool calls with name, parameters, result, and duration                                 |
| `state`         | string         | Context graph state at this turn                                                       |
| `inner_thought` | string or null | Agent's internal reasoning (when available)                                            |
| `latency_ms`    | integer        | Response latency for this turn                                                         |
| `barge_in`      | object or null | Barge-in details if the agent was interrupted (interrupted text, discarded utterances) |

### Prompt Trace

```
GET /agent/calls/{call_id}/prompts
```

Captured LLM prompt data can include the system prompt, conversation history, tool definitions, and model response for a turn. Availability depends on runtime capture and retention, and the data is intended for debugging unexpected agent behavior.

### Call Trace Analysis

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/calls/{call\_id}/trace-analysis" method="get" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

Audio-native intelligence analysis computed asynchronously after call completion. See [Call Trace Analysis](https://docs.concurrence.com/developer-guide/platform-api/safety/call-trace-analysis.md) for the full endpoint reference.

Returns `status: pending` while analysis is in progress and `status: unavailable` if analysis is not possible for this call.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.concurrence.com/developer-guide/platform-api/conversations/voice-agent.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
