> For the complete documentation index, see [llms.txt](https://docs.concurrence.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.concurrence.com/operations-and-safety/runtime-safety.md). # Runtime Safety Current runtime safety controls, their enforcement boundaries, operator intervention, and evidence-based release practices. Runtime safety is a layered system. Authored conversation policy guides the agent, structural controls constrain supported execution paths, server-side authorization limits writes, and operators can intervene through the unified Runs surface. The current architecture does not depend on a separate semantic monitoring or regulatory-triage service. Safety-critical behavior must be expressed in the agent's context graph, tool permissions, escalation instructions, and operating procedures, then verified against representative scenarios.
Authored policy guides reasoning. Supported execution paths enforce structural and authorization controls. Voice shadow observers provide diagnostic evidence without changing the live call. Authorized operators use channel-specific intervention controls.

These are distinct responsibilities, not a universal sequence of safety gates. The control map specifies each runtime’s scope.

## Control Map | Control | Scope | Current Behavior | | --------------------------------------- | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Context graph policy** | Voice and text | States define objectives, available tools, exit conditions, guardrails, and boundary constraints that shape agent behavior. | | **Graph navigation invariants** | Platform-managed context graph runtime | Navigation resolves to an engageable state, detects loops, and caps per-turn transitions in code. | | **Cross-runtime invariant observation** | Voice | The platform records shadow verdicts for engageable-state, loop, and iteration checks. These verdicts do not change the live call. | | **Response guardrail observation** | Voice | Candidate responses are compared with the active state's guardrails and boundary constraints in shadow mode. Verdicts are diagnostic only. | | **Integration approval policy** | Text conversations | A gated endpoint or an `approval_required` skill waits for an authorized approve or reject decision before it runs. Other surfaces don't offer these tools; see [Integration Tool Approval](#integration-tool-approval). | | **Write scope** | Supported agent write tools | A server-side write scope binds model-originated writes to an allowed workspace and entity; each registered tool constrains its resource shape and source-appropriate confidence. | | **In-flight write deduplication** | Supported voice runtimes | An identical write-tool call is short-circuited while the first invocation is still running. | | **Unknown write outcomes** | Voice and text | An integration write that got no answer saying whether it went through returns `"outcome": "unknown"`, and the platform refuses to send it again in that conversation. See [Endpoint Retries](https://docs.concurrence.com/developer-guide/platform-api/integrations#endpoint-retries). | | **Caller verification** | Voice and text | A caller matched by phone number or `entity_id` stays unverified, with the patient record out of the agent's context, until their full name and date of birth match one record through `world_confirm_patient_identity`. Built-in record reads then return only the caller's own record or a household member's that names them; see [Caller Verification](https://docs.concurrence.com/developer-guide/platform-api/conversations/voice-agent#caller-verification). On a service with `platform_caller_verification: false`, the platform never asks its callers to verify, its built-in record reads refuse with `verification_off`, and the agent is told not to share anything from a patient record. | | **Call-opening notice** | Platform-managed voice and `gpt_realtime` | Calls open with an AI and recording notice. When it plays depends on the runtime and the call's direction; see [Call Opening](https://docs.concurrence.com/developer-guide/platform-api/conversations/voice-agent#call-opening). | | **Transfer destinations** | Voice | A dynamic transfer dials only a number listed in the service's `approved_destinations`; any other number is refused without dialing. | | **Crisis resource guidance** | Text, platform-managed voice, and `gpt_realtime` | The agent's instructions always carry the deployment region's crisis resource and tell it to give that resource on suicidal thoughts, self-harm, or an overdose, and on `gpt_realtime` at the first hint. This is model guidance, not a deterministic check. | | **Self-harm script on text** | Text | A message that matches the self-harm patterns, in English or Spanish, is answered with an authored script that gives the region's crisis line where one exists and the emergency number, and the conversation stays open. Pattern matching can miss a phrasing; the crisis guidance above still applies. | | **Emergency script on `gpt_realtime`** | `gpt_realtime` | A caller line recognized as describing a medical emergency or self-harm, in English or Spanish, cuts off any reply the agent is speaking outside a warm transfer's briefing, and the agent speaks an authored script first: for a medical emergency, to hang up and call the region's emergency number now; for self-harm, the region's crisis line and the emergency number for immediate danger, or the emergency number to call now where the region has no crisis line. The realtime model speaks the script from the authored text, so its wording can vary. Recognition covers a limited set of phrasings and can miss others, or play the script when a caller only mentions a condition; for a missed line about suicidal thoughts, self-harm, or an overdose, the crisis guidance above still applies, and any other missed emergency rests on the agent's own instructions. On platform-managed voice, the agent's own reply plays. See [Realtime Emergency Script](https://docs.concurrence.com/developer-guide/platform-api/conversations/voice-agent#realtime-emergency-script). | | **Emergency calls stay open** | Platform-managed voice and `gpt_realtime` | Once the caller describes a medical emergency or self-harm, in English or Spanish, the agent doesn't end the call: on the platform-managed runtime a terminal Context Graph state doesn't hang up, on `gpt_realtime` the model's `end_call` is refused, and on both the last silence leaves the line open. The call ends when the caller hangs up or at the call's duration limit. A `gpt_realtime` service with `medical_emergency_ending: end_after_script` instead ends the call after a medical emergency described in English, once the script and a closing line have played; self-harm keeps the call open in every setting. The safe exit can still end it, as can an operator who has taken the call on the platform-managed runtime; see [Terminal State & Auto-Hangup](https://docs.concurrence.com/developer-guide/platform-api/conversations/voice-agent#terminal-state-and-auto-hangup). Recognition can miss a phrasing. | | **Tool and knowledge-base content** | Text, platform-managed voice, and `gpt_realtime` | Tool results, including knowledge-base articles and records from your systems, reach the model marked as untrusted content. The agent is instructed to answer from it and not to follow instructions inside it, such as ones that try to change its role, trigger a tool call or a transfer to a number the content gives, reveal information, advise on medication, or replace an official phone number, link, or payment detail. This is model guidance, not a deterministic check. | | **Spoken-reply check** | Platform-managed voice and `gpt_realtime` | A pattern check and a model check review each reply on its way to the caller, and a reply they stop is cut off and followed by a short correction. See [Reply Checks](#reply-checks). | | **Text reply check** | Text | A pattern check reviews each reply before it is sent on any channel, and a reply it stops is replaced by a short correction. See [Reply Check](https://docs.concurrence.com/developer-guide/platform-api/conversations#reply-check). | | **Speech recognition failure** | Platform-managed voice | A recognizer that fails and can't reconnect is replaced once by another for the caller's language. Otherwise the call takes the safe exit below; see [Graceful Degradation](https://docs.concurrence.com/developer-guide/platform-api/conversations/voice-agent#graceful-degradation). | | **Safe exit** | Platform-managed voice and `gpt_realtime` | When a platform-managed call can't recover its speech recognition or can't set up its agent, a caller's turn gets no reply on either runtime, or a `gpt_realtime` call can't start, because something it needs fails before its realtime session starts or its realtime session doesn't open (its connection fails or times out, its session setup can't be sent, or the realtime model doesn't answer the setup or refuses it), the agent plays fixed lines rather than generated ones: the AI and recording notice if the caller hasn't heard all of it, an apology, and a transfer to the service's fixed forwarding number, or a closing line and the end of the call. When a call's setup fails after its service configuration has loaded, the lines play in Spanish for an agent whose default language is Spanish, in English then Spanish for a multilingual agent, and in English for an agent in any other language. If the service has a fixed forwarding number, the call also transfers to it. A call whose service configuration can't be loaded when it arrives plays in English and isn't handed to that number. In an emergency, the closing line tells the caller to hang up and call 9-1-1 or their local emergency number. The lines are fixed audio prepared ahead of time: every `gpt_realtime` exit plays that audio, and a platform-managed exit plays it unless the call has prepared the lines in its own voice, in the language it speaks. | | **Request for a person** | Text, platform-managed voice, and `gpt_realtime` | On text and platform-managed voice, a request for a person that the pattern check recognizes doesn't end the conversation, and on `gpt_realtime` the agent is instructed the same way. The agent is told to transfer when it can, to take a message only through a tool that records one, and otherwise to say how to reach the team; see [Safety & Escalation](https://docs.concurrence.com/developer-guide/platform-api/conversations/voice-agent#safety-and-escalation). | | **Operator control** | Eligible live runs | Authorized operators can listen, take over, send guidance, author a text turn, and hand control back, depending on channel. | This map distinguishes controls that actively enforce behavior from controls that only produce evidence. That distinction matters when setting clinical and operational expectations. ### Reply Checks Every text channel and both voice runtimes check the agent's replies: text and platform-managed voice before the person gets them, and `gpt_realtime` while they play. * **What they stop.** A pattern check, on every runtime, stops dosing or medication-change advice and a promise to contact emergency services. It also stops a booking, cancellation, move, or message claimed as done, and a promise to book, reschedule, cancel, or act right away, when no matching write backs it. On both voice runtimes, a model check also stops a diagnosis, record details shared with a caller who hasn't verified, and other actions claimed as done that didn't succeed on the call. A result from a data query or integration endpoint that declares it reads no patient record, such as an FAQ, a location, or opening hours, doesn't count as a patient record in the call, so while nothing else in the call has brought one in, the model check doesn't treat a public answer as record details; see [Endpoint Read Declarations](https://docs.concurrence.com/developer-guide/platform-api/integrations#endpoint-read-declarations). * **What backs a claim.** A claim that something is done needs a matching write that succeeded: earlier in the call on voice, or in the same turn on text. On voice, a promise to do something now is backed when the same reply calls the matching write. On text, a promise needs a write that already succeeded in that turn. Only a write that declares what it does backs a claim, and a write whose outcome is unknown backs none; see [Endpoint Write Declarations](https://docs.concurrence.com/developer-guide/platform-api/integrations#endpoint-write-declarations). * **On platform-managed voice,** each sentence passes the pattern check before the caller hears it, and the model check too when it answers in time; the first sentence can wait briefly for it. A stopped sentence keeps the words already spoken, drops the rest of the reply and its tool calls, and plays a short correction. * **On `gpt_realtime`,** the checks review the reply while it plays, so the caller can hear some or all of a sentence before a stop lands. A stopped reply is cut off, its tool calls don't run, except a transfer it had already requested in full, and the agent speaks a short correction. A stop that lands after the caller has spoken again cuts nothing: the agent is told what was wrong, and the caller hears a correction only for dosing or medication advice, a diagnosis, or a promise to contact emergency services. * **On a warm transfer,** both voice runtimes also check the agent's briefing, which the caller and the destination both hear. A stopped briefing is cut off like any other reply, and the correction plays before the hand-off when the stop lands in time. On `gpt_realtime`, the hand-off can wait a moment for the check to finish with the briefing. * **On text,** the pattern check reviews the reply before it is sent, and a streamed turn previews it a sentence at a time. A stopped reply is replaced whole: the preview stops at that sentence, and the correction goes out in `output` and the `message` event. After the person describes an emergency, the correction is the emergency script, a short line that points them to emergency help. When a reply claims a write whose outcome is unknown, the correction says the agent can't confirm it went through. On its next turn, the agent is told what was replaced and why. Have the agent call a tool before it says what the tool did, and test your confirmation wording: either check can miss a phrasing or stop a correct reply.
A pattern check runs on every runtime, and both voice runtimes add a model check. Platform-managed voice checks each sentence before the caller hears it, gpt_realtime checks the reply as it plays, and text checks it before it is sent; a stopped reply ends with a short correction.

Pattern checks cover English and Spanish.

## Safety in the Agent Policy The agent evaluates the conversation within its normal reasoning loop. The platform-level checks in the control map cover a few narrow cases, but they do not compensate for missing escalation instructions or incomplete boundaries. For each safety-sensitive state, define: * **Objective** - What the agent is trying to accomplish in that state. * **Allowed tools** - The smallest tool set needed for the task. * **Boundary constraints** - What the agent must not claim, recommend, or attempt. * **Escalation conditions** - Observable situations that require a human or a configured transfer target. * **Safe fallback** - What the agent should say and do if the required person, tool, or destination is unavailable. * **Exit conditions** - The evidence needed before the graph advances. On the Platform API, reusable policy that spans states should be expressed through shared context graph structure, state guidance, transitions, eligible tools, and tested authoring conventions. Classic dynamic behavior sets are a separate Classic API capability and are not a Platform runtime control. {% hint style="warning" %} Safety instructions are model-facing policy. They are not a substitute for server-side authorization, clinical governance, or a staffed escalation process. {% endhint %} ## Structural Navigation Controls The platform-managed context graph navigator enforces three structural properties during a turn: 1. **Engageable landing** - Routing states are resolved until the conversation reaches a state that can handle the next user interaction. 2. **Loop handling** - Repeated navigation is detected so the engine does not traverse the graph indefinitely. 3. **Transition cap** - A bounded number of state changes can occur within one turn. If navigation cannot resolve safely, this runtime stays on the last engageable state rather than presenting a routing-only state to the caller. The voice layer also observes these properties across supported runtimes. That observer is shadow-only: it records diagnostic verdicts but does not interrupt, reroute, or end a call. Runtimes that navigate independently therefore do not inherit the platform-managed navigator's enforcement merely because they produce the same telemetry. ## Guardrail Evaluation Status State guardrails and boundary constraints are included in the agent's active instructions. Separately, the platform runs a lightweight response-text evaluator on voice calls and records whether a candidate response appears to overlap a configured rule. {% hint style="warning" %} The response evaluator is shadow-only. A recorded `block` verdict does not block or rewrite the response, and evaluator failure does not affect the call. Treat these verdicts as calibration and regression evidence, not as an enforced clinical safety boundary. {% endhint %} Because shadow evaluation observes text rather than clinical truth, teams should review false positives and false negatives before using its output in an operational process. ## Write Safety ### Integration Tool Approval Workspace integrations can set an approval policy of `none`, `writes`, or `all`, and a skill can be marked `approval_required`. In a text conversation, a call to a gated endpoint or an approval-required skill is parked before it runs until an authorized reviewer approves or rejects it. The default policy is `none`. This mechanism is conversation-scoped and fail-closed. Only a text conversation can pause a tool call for a decision, so voice calls, simulations, voice notes, companion skills, and trigger actions don't get these tools. See [Approval-Gated Integration Writes](https://docs.concurrence.com/developer-guide/platform-api/integrations/approval-gated-integration-writes#key-concepts). ### Agent Write Scope Supported agent write tools receive a server-constructed write scope that restricts which workspace and entity the operation can target. The registered tool contract separately constrains the resource shape and assigns or caps the confidence available to model-originated data. These checks protect the persistence boundary even if the model supplies an unexpected identifier. System-originated projection work follows separate trusted-service authorization and is not evidence that an agent can bypass its own scope. ### Duplicate Calls Supported voice runtimes track identical write-tool calls that are already in progress. If the model repeats the same write before the first completes, the duplicate is rejected as already running. Once the first finishes, a later identical call can run again, unless the first was an integration write whose outcome is unknown, which isn't sent again in that conversation. [Endpoint Retries](https://docs.concurrence.com/developer-guide/platform-api/integrations#endpoint-retries) lists the failures after which the platform retries an integration write. None of this is a permanent idempotency guarantee. Operations that need durable idempotency should still use the operation's documented idempotency key or destination-specific protection. ### External Delivery Recording an observation in the world model does not confirm an external mutation, and the world model does not deliver observations to external systems. An external mutation happens through a tool the agent calls, such as an integration endpoint, and its result is the evidence of what the destination accepted. ## Human Intervention The unified Runs surface gives operators one place to monitor eligible live work: * **Voice** - Join in listen mode, explicitly take over, send guidance, switch mode, and hand the call back. * **Text channels** - Pause an eligible run, author the next turn, and hand control back. * **History** - Inspect the transcript, run metadata, available analysis, and recorded operator actions. Operator actions require the Operator update permission, available to operator, admin, and owner roles. Supported action routes bind the requested operator identity to the authenticated caller and emit operator-attributed events for takeover-related actions. An authored escalation policy must also define what happens when no operator is available. Appropriate fallbacks can include a configured transfer target, callback workflow, emergency instruction, or a bounded refusal. Do not assume the platform automatically pages a clinician or creates a post-call review item. ## Validation Before Release Simulations exercise the current workspace configuration through the real reasoning and tool path. Simulation-originated world-model writes are tagged so production analytics can exclude them, but simulations do not create an isolated configuration branch and do not replace every external dependency with a stub. Use dedicated test workspaces, test entities, or safe integration environments when a scenario could reach a real external system. Separate broad text-tier validation from real-audio tests that measure transcription, speech rendering, turn timing, and media attachment. A practical release review includes: 1. Run targeted cases for each safety boundary and escalation path. 2. Run regression suites against the configuration intended for release. 3. Inspect normalized checks, transcripts, tool calls, and failure reasons. 4. Review shadow guardrail and control-invariant evidence where available. 5. Confirm operator staffing and fallback procedures. 6. Deliberately update the pinned `release` version set when the evidence is acceptable. Simulation results and drift analysis do not automatically block version-set promotion or roll back a release. Promotion and rollback are deliberate operator actions. ## Production Review After release, use conversation evaluations, run history, integration call results, and operational metrics to find changes in behavior. Compare defined cohorts and stable time windows rather than treating a single aggregate score as proof of safety. When a regression appears: 1. Identify affected channels, services, versions, and cohorts. 2. Add or update a reproducible simulation case. 3. Verify whether the issue is agent behavior, tool behavior, media quality, or destination failure. 4. Correct the smallest responsible configuration or integration boundary. 5. Re-run the affected suite before moving the release pin. ## What Runtime Safety Does Not Guarantee * **Perfect outcomes** - Model behavior and external systems remain fallible. * **Automatic clinical triage** - The crisis resource guidance, the self-harm and emergency scripts, and the voice and text reply checks cover narrow cases. Organizations must author, test, and staff their own clinical escalation protocols. * **Enforced shadow verdicts** - Guardrail and cross-runtime invariant observers record evidence only. * **Universal human review** - Only gated integration endpoints and approval-required skills in text conversations use an approval path. * **Automatic deployment rollback** - Teams decide when to move a version-set pin. * **Automatic outbound reconciliation** - Verify each integration call's result at the destination. {% hint style="info" %} Triage and safety evaluation support operational decisions; they do not replace clinical assessment. Organizations remain responsible for clinical protocols, escalation staffing, destination verification, and regulatory review. {% endhint %} ## Related Documentation * [Context Graphs](/agent/context-graphs.md) * [Operators](/operations-and-safety/operators.md) * [Testing and Evaluation](/testing/testing.md) * [Connectors and EHR](/data/connectors-and-ehr.md) * [Compliance](/operations-and-safety/compliance.md) * [Safety and monitoring APIs](https://docs.concurrence.com/developer-guide/platform-api/safety) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.concurrence.com/operations-and-safety/runtime-safety.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.