Simulation Coverage
Explore context graph coverage, fork and score text simulations, aggregate per-conversation verdicts, and run saved-case benchmarks with tools in simulation mode.
The simulation coverage API provides branch-and-bound exploration of context graph state space. It replaces random sampling with targeted exploration that steers conversations toward untested states, tools, and transitions.
Core Model
Coverage is organized into three levels:
Coverage Run
A campaign that contains simulation sessions, whose tools run in simulation mode.
Coverage Session
A conversation path within a run. Sessions can be forked from other sessions at any turn.
Coverage Turn
A single agent turn within a session, recording context graph state, tool calls, and scores.
Sessions form a tree structure through the fork mechanism. Each session tracks its parent_session_id and fork_turn_index, so the full exploration tree is reconstructable from the session list.
Endpoints
All endpoints are scoped to a workspace and require a workspace API key.
Create Coverage Run
Create a simulation coverage run with an optional Lakebase branch.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Successful Response
Validation Error
POST /v1/{workspace_id}/simulations/runs HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 90
{
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"branch_name": "text",
"tags": [
"text"
]
}{
"id": "123e4567-e89b-12d3-a456-426614174000",
"workspace_id": "123e4567-e89b-12d3-a456-426614174000",
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"branch_name": "text",
"status": "text",
"tags": [
"text"
]
}Returns the created run with ID, status (running), tags, and optional branch label metadata. The label does not create a separate database branch.
Run and session tags that start with suite:, suite_run:, or safety: are set by the server, and sending one returns 422. A workspace can have 10 simulation launches running at once, and a case, suite, benchmark, or bridge launch beyond that returns 429.
Create Coverage Session
Creates a new conversation session within a run. The session starts with a fresh agent state. A run that does not exist in the workspace returns 404, and a service_id that is not the run's service returns 422.
Create a simulation session within a run. Proxies to agent-engine.
The run must be in the workspace (404 otherwise) and service_id must be
the run's service (422 otherwise), both checked before agent-engine is called.
entity_id ownership: agent-engine's _resolve_caller queries the entity with workspace_id scoping — a workspace-A entity_id resolves to None in workspace-B, so no cross-tenant data is exposed.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Simulated caller phone number used to resolve patient context when entity_id is not provided (or does not match a row). Omitted/blank values are normalized to the sim-orchestrator sentinel, which yields no phone match and an empty caller context — same default the voice test-call WebSocket uses, so patient lookups resolve consistently across modalities.
Optional world entity UUID to bind caller context directly. Precedence rules: (1) when this resolves to a world entity in the workspace, it WINS — caller_id is not used for identity resolution, and the resolution provenance is entity_id_lookup; (2) when this is a well-formed UUID with no matching entity (stale, deleted, wrong workspace), the session falls back to phone lookup against caller_id (provenance phone_lookup) — no error is raised, the sim is survivable; (3) malformed UUIDs are rejected with HTTP 422 before the request reaches agent-engine. The supplied caller_id is still recorded on the active call and surfaced in the greeting metadata in case (1).
Successful Response
Run or simulation configuration not found
Request validation or simulation configuration error
Simulation runtime error
Voice agent not reachable
POST /v1/{workspace_id}/simulations/runs/{run_id}/sessions HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 160
{
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"branch_name": "text",
"tags": [
"text"
],
"caller_id": "text",
"entity_id": "123e4567-e89b-12d3-a456-426614174000"
}{
"session_id": "text",
"greeting": "text",
"is_terminal": true,
"snapshot": {
"current_state": {
"name": "text",
"type": "text",
"objective": "",
"actions": [
{
"ANY_ADDITIONAL_PROPERTY": "anything"
}
],
"exit_conditions": [
{
"ANY_ADDITIONAL_PROPERTY": "anything"
}
],
"action_guidelines": [
"text"
],
"boundary_constraints": [
"text"
],
"guardrails": [
{
"ANY_ADDITIONAL_PROPERTY": "anything"
}
],
"tools": [
"text"
],
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"reachable_states": [
{
"ANY_ADDITIONAL_PROPERTY": "anything"
}
],
"turn_policy": {
"barge_in_enabled": true,
"greeting_shield_s": 0,
"safety_response": "suspend_forward",
"context_strategy": "full",
"block_forward_call": false,
"block_forward_call_after_turns": -1
},
"conversation_history": [
{
"ANY_ADDITIONAL_PROPERTY": "anything"
}
],
"states_visited": [
"text"
],
"state_transitions": [
[
"text"
]
],
"total_turns": 0,
"tools_called": [
"text"
],
"terminal_reached": false,
"context_graph_topology": [
{
"ANY_ADDITIONAL_PROPERTY": "text"
}
],
"terminal_state": "text",
"ANY_ADDITIONAL_PROPERTY": "anything"
}
}Step a Session
Sends a simulated user message and records the agent's response as a turn.
Step the session in the path and auto-store the turn observation.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
One caller turn for the session named in the path.
Leave unset: the path names the session. A value that differs from the path returns 422.
Successful Response
Simulation session or configuration not found
Request validation or simulation configuration error
Simulation runtime error
Voice agent not reachable
POST /v1/{workspace_id}/simulations/sessions/{session_id}/step HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 71
{
"caller_text": "text",
"emotion": "text",
"valence": 1,
"session_id": "text"
}{
"observation": {
"state_before": "text",
"state_after": "text",
"state_changed": true,
"agent_text": "text",
"is_terminal": true,
"tools_called": [
"text"
],
"tool_call_details": [
{
"tool_name": "text",
"input": "",
"result": "",
"call_id": "",
"succeeded": true
}
],
"selected_action": "text",
"empathy_tier": 0,
"has_pause": false,
"engine_ms": 1,
"nav_ms": 1
},
"snapshot": {
"current_state": {
"name": "text",
"type": "text",
"objective": "",
"actions": [
{
"ANY_ADDITIONAL_PROPERTY": "anything"
}
],
"exit_conditions": [
{
"ANY_ADDITIONAL_PROPERTY": "anything"
}
],
"action_guidelines": [
"text"
],
"boundary_constraints": [
"text"
],
"guardrails": [
{
"ANY_ADDITIONAL_PROPERTY": "anything"
}
],
"tools": [
"text"
],
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"reachable_states": [
{
"ANY_ADDITIONAL_PROPERTY": "anything"
}
],
"turn_policy": {
"barge_in_enabled": true,
"greeting_shield_s": 0,
"safety_response": "suspend_forward",
"context_strategy": "full",
"block_forward_call": false,
"block_forward_call_after_turns": -1
},
"conversation_history": [
{
"ANY_ADDITIONAL_PROPERTY": "anything"
}
],
"states_visited": [
"text"
],
"state_transitions": [
[
"text"
]
],
"total_turns": 0,
"tools_called": [
"text"
],
"terminal_reached": false,
"context_graph_topology": [
{
"ANY_ADDITIONAL_PROPERTY": "text"
}
],
"terminal_state": "text",
"ANY_ADDITIONAL_PROPERTY": "anything"
}
}Returns the turn observation including agent response, context graph state, tool calls, and turn index. The step uses the session in the path. A body session_id that names a different session returns 422. A runtime failure or timeout returns 502, and an unreachable runtime returns 503.
Recommend Caller Messages
Suggests up to n candidate caller messages (default 5, maximum 20) for the session's current state. Use them as inputs to step or fork. Suggestions are model-generated, so review them rather than treating them as a coverage guarantee.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
5Successful Response
LLM-generated caller message suggestions
Simulation session or configuration not found
Request validation or simulation configuration error
Simulation runtime error
Voice agent not reachable
POST /v1/{workspace_id}/simulations/sessions/recommend HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 27
{
"session_id": "text",
"n": 5
}{
"suggestions": [
"text"
]
}The response carries suggestions, a list of message strings.
Fork a Session
Atomically clones the parent session state N times, steps each clone with a different simulated message, and returns all observations. This is the core branch-and-bound primitive.
Fork a session into N branches — clone + step each alternative atomically.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Successful Response
One branch per alternative, in request order.
The parent's turn count when it forked. Each branch's first turn is stored at the next index.
Simulation session or configuration not found
Request validation or simulation configuration error
Simulation runtime error
Voice agent not reachable
POST /v1/{workspace_id}/simulations/sessions/{session_id}/fork HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 70
{
"alternatives": [
{
"caller_text": "text",
"emotion": "text",
"valence": 1
}
]
}{
"parent_session_id": "text",
"fork_turn_index": 1,
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"branches": [
{
"session_id": "text",
"observation": {
"state_before": "text",
"state_after": "text",
"state_changed": true,
"agent_text": "text",
"is_terminal": true,
"tools_called": [
"text"
],
"tool_call_details": [
{
"tool_name": "text",
"input": "",
"result": "",
"call_id": "",
"succeeded": true
}
],
"selected_action": "text",
"empathy_tier": 0,
"has_pause": false,
"engine_ms": 1,
"nav_ms": 1,
"caller_text": "text"
},
"error": "text"
}
]
}Each child session inherits the parent's full conversation history up to the fork point. The parent session remains unchanged.
Promote a Session
Interactive playground sessions are created without a coverage run, so run-scoped operations like fork and score are unavailable on them. Promotion binds a run-less session to a newly created coverage run, which unblocks fork and score. This lets developers iterate in the playground and then promote interesting sessions for deeper analysis without re-running the conversation.
Promote a run-less (interactive playground) session into a coverage run.
Interactive text sessions are created run-less, so fork — which requires
a run_id — 404s on them. This creates a coverage run, binds the existing
agent-engine session to it (adopt-run, a Valkey meta update — the session
itself is not recreated), and writes the platform-api SimulationCoverageSession
record that get_session_metadata returns, unblocking fork/score.
Idempotent: a session that already has a coverage record returns its existing
run with already_bound=True.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Successful Response
Result of binding a run-less session to a (newly created) coverage run.
The coverage run the session is now bound to
The promoted session id (unchanged)
True when the session was already run-scoped; the existing run is returned unchanged
Simulation session or configuration not found
Request validation or simulation configuration error
Simulation runtime error
Voice agent not reachable
POST /v1/{workspace_id}/simulations/sessions/{session_id}/promote HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"session_id": "text",
"already_bound": true
}The response returns the session_id (unchanged), the run_id of the coverage run the session is now bound to, and already_bound. Promotion is idempotent: a session that was already run-scoped returns already_bound: true with its existing run_id unchanged.
Score a Session
Evaluates the session against configured metrics. Scores are attributed per-session, not per-state-visit.
ASI assigns a score to a completed simulation session.
404 when the workspace has no coverage session with this id, such as a run-less playground session that was never promoted.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Successful Response
Simulation session not found
Validation Error
POST /v1/{workspace_id}/simulations/sessions/{session_id}/score HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 36
{
"score": 1,
"score_rationale": "text"
}{
"ANY_ADDITIONAL_PROPERTY": "text"
}Returns a confirmation status ({ "status": "scored" }). The supplied score and rationale are persisted; this endpoint does not return computed metric scores. Aggregated metric and coverage scores are read back through the coverage graph, the session list, or the benchmark results endpoint.
Get Coverage Graph
Returns the knowledge graph for the run. The graph has two layers:
Observed turns
Aggregated per context graph state: session count, pass rate, total turns
Topology overlay
Ghost nodes for context graph states that exist in the state machine but were never reached. These represent coverage gaps.
Get the B&B knowledge graph for a service — topology + observations merged.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
falseSuccessful Response
The service's context graph states and transitions, overlaid with the stored turns.
Delayed-metric availability for the run. Set only when the request names a run_id.
Validation Error
GET /v1/{workspace_id}/simulations/services/{service_id}/graph HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"total_sessions": 1,
"total_turns": 1,
"total_forks": 1,
"coverage_percentage": 1,
"state_nodes": [
{
"id": "text",
"type": "text",
"theoretical": true,
"observed": true,
"turn_count": 1,
"session_count": 1,
"self_loop_count": 1,
"transition_out_count": 1,
"pass_count": 1,
"fail_count": 1,
"unknown_count": 1,
"is_terminal": true,
"first_seen_turn": 1,
"tools_used": [
"text"
],
"average_empathy_tier": 1,
"average_engine_ms": 1,
"average_nav_ms": 1
}
],
"edges": [
{
"source": "text",
"target": "text",
"type": "transition",
"theoretical": true,
"observed": true,
"frequency": 1,
"first_seen_turn": 1
}
],
"tree": {
"root_sessions": [
"text"
],
"forks": [
{
"parent_session_id": "text",
"fork_turn_index": 1,
"children": [
"text"
]
}
]
},
"sessions": [
{
"metric_status": "pending",
"metric_result_count": 1,
"metrics_last_checked_at": "2026-01-01T00:00:00.000Z",
"immediate_score": 1,
"immediate_score_rationale": "text",
"session_id": "text",
"parent_session_id": "text",
"fork_turn_index": 1,
"tags": [
"text"
],
"total_turns": 1,
"terminal_reached": true,
"score": 1,
"created_at": "text"
}
],
"turns": [
{
"session_id": "text",
"turn_index": 1,
"state_before": "text",
"state_after": "text",
"state_changed": true,
"caller_text": "text",
"agent_text": "text",
"tools_called": [
"text"
],
"tool_call_details": [
{
"tool_name": "text",
"input": "",
"result": "",
"call_id": "",
"succeeded": true
}
],
"empathy_tier": 1,
"is_terminal": true,
"created_at": "text",
"global_index": 1
}
],
"metric_status": "pending",
"metric_result_count": 1,
"metrics_last_checked_at": "2026-01-01T00:00:00.000Z"
}Complete a Coverage Run
Marks the run as finished. Sessions and turns remain available for read access. A run that does not exist returns 404, and a run that already finished returns 409.
Finish a running run: failed when error is set, otherwise completed.
A run finishes once. Completing a finished run returns 409 and leaves it as it was.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Failure reason. Set it to finish the run as failed.
Successful Response
Run not found
Run has already finished
Validation Error
POST /v1/{workspace_id}/simulations/runs/{run_id}/complete HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"status": "completed"
}Get Coverage Run
Returns the full run payload, including the stored bridge_request (original request body) and scenarios (generated persona/scenario list) for runs created through the Simulation Bridge. The list runs endpoint omits these fields to keep responses compact.
The response also includes a checks array alongside the raw eval_results - a normalized, read-only projection that flattens each metric and assertion verdict into one uniform shape. Per-case assertion detail responses carry the same checks array. Use it for consistent rendering; the raw fields are unchanged.
Fetch a single run including objective / bridge_request / scenarios.
Use the bridge_request payload to re-POST /bridge and replay the run.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Successful Response
Full run payload — used by GET /runs/{run_id} for replay flows.
00pendingPossible values: 00000Validation Error
GET /v1/{workspace_id}/simulations/runs/{run_id} HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"id": "123e4567-e89b-12d3-a456-426614174000",
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"branch_name": "text",
"status": "running",
"error": "text",
"tags": [
"text"
],
"total_sessions": 0,
"total_turns": 0,
"objective": "text",
"bridge_request": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"scenarios": [
{
"ANY_ADDITIONAL_PROPERTY": "anything"
}
],
"snapshot": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"total_scenarios_requested": 1,
"created_at": "text",
"completed_at": "text",
"metric_status": "pending",
"metric_result_count": 0,
"metrics_last_checked_at": "2026-01-01T00:00:00.000Z",
"eval_results": [
{
"id": "123e4567-e89b-12d3-a456-426614174000",
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"case_id": "123e4567-e89b-12d3-a456-426614174000",
"session_id": "text",
"eval_key": "text",
"eval_type": "assertion",
"assertion_kind": "text",
"metric_key": "text",
"status": "passed",
"passed": true,
"score": 1,
"expected": null,
"actual": null,
"rationale": "text",
"justification": "text",
"references": [
1
],
"details": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"definition": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"created_at": "text"
}
],
"eval_result_count": 0,
"eval_pass_count": 0,
"eval_fail_count": 0,
"eval_error_count": 0,
"checks": [
{
"source_type": "metric",
"key": "text",
"label": "text",
"value_type": "numeric",
"evaluation_method": "threshold",
"expected": null,
"actual": null,
"expected_display": "text",
"actual_display": "text",
"verdict": "passed",
"score": 1,
"rationale": "text",
"references": [
1
],
"run_id": "text",
"case_id": "text",
"session_id": "text",
"trace_id": "text"
}
]
}Get Simulation Performance
The performance operation aggregates recent graded case and suite runs into one overview. Use service_id to restrict the result and limit to bound the recent runs analyzed.
pass_rate is passed divided by passed plus failed. Grading errors, pending, and skipped verdicts are counted on every rollup (error, pending, and skipped) but stay out of the rate, and the rate is null when there is no passed or failed verdict. LLM-judge verdicts carry passed, rationale, and references with no score, and checks entries no longer include score_label. Metric checks keep score.
When metrics[].checks is present, use it for a consistent display across metric and assertion verdicts. It normalizes the source, key, label, verdict, expected and actual values, optional score, rationale, references, and run/session identifiers while preserving results for callers that need the raw evaluation records. Do not require either metric-detail field unless the OpenAPI schema for your assigned deployment exposes it; deployments still on the compact response return only metric_key, label, and per_run.
Aggregate recent graded (case/suite) runs into the Simulations overview: overall pass rate, per-metric trend series, and per-case / per-suite rollups (each with its per-conversation verdicts).
limit (1 to 50) bounds how many recent runs are read. One request
replaces the console's former one-fetch-per-run fan-out.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
15Successful Response
Aggregated Simulations overview: one request in place of the console's former one-detail-fetch-per-run fan-out.
Validation Error
GET /v1/{workspace_id}/simulations/performance HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"overview": {
"overall_pass_rate": 1,
"evals_run": 0,
"error_count": 0,
"pending_count": 0,
"skipped_count": 0,
"needs_attention_count": 0,
"runs_analyzed": 0
},
"metrics": [
{
"metric_key": "text",
"label": "text",
"per_run": [
{
"at": "text",
"value": 1,
"passed": 0,
"measured": 0
}
],
"results": [
{
"id": "123e4567-e89b-12d3-a456-426614174000",
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"case_id": "123e4567-e89b-12d3-a456-426614174000",
"session_id": "text",
"eval_key": "text",
"eval_type": "assertion",
"assertion_kind": "text",
"metric_key": "text",
"status": "passed",
"passed": true,
"score": 1,
"expected": null,
"actual": null,
"rationale": "text",
"justification": "text",
"references": [
1
],
"details": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"definition": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"created_at": "text"
}
],
"checks": [
{
"source_type": "metric",
"key": "text",
"label": "text",
"value_type": "numeric",
"evaluation_method": "threshold",
"expected": null,
"actual": null,
"expected_display": "text",
"actual_display": "text",
"verdict": "passed",
"score": 1,
"rationale": "text",
"references": [
1
],
"run_id": "text",
"case_id": "text",
"session_id": "text",
"trace_id": "text"
}
]
}
],
"by_case": [
{
"case_id": "123e4567-e89b-12d3-a456-426614174000",
"case_name": "text",
"pass_rate": 1,
"passed": 0,
"failed": 0,
"error": 0,
"pending": 0,
"skipped": 0,
"measured": 0,
"latest_run_id": "123e4567-e89b-12d3-a456-426614174000",
"latest_run_at": "text",
"assertions": [
{
"eval_key": "text",
"eval_type": "assertion",
"passed": 0,
"failed": 0,
"error": 0,
"pending": 0,
"skipped": 0,
"pass_rate": 1,
"results": [
{
"id": "123e4567-e89b-12d3-a456-426614174000",
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"case_id": "123e4567-e89b-12d3-a456-426614174000",
"session_id": "text",
"eval_key": "text",
"eval_type": "assertion",
"assertion_kind": "text",
"metric_key": "text",
"status": "passed",
"passed": true,
"score": 1,
"expected": null,
"actual": null,
"rationale": "text",
"justification": "text",
"references": [
1
],
"details": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"definition": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"created_at": "text"
}
],
"checks": [
{
"source_type": "metric",
"key": "text",
"label": "text",
"value_type": "numeric",
"evaluation_method": "threshold",
"expected": null,
"actual": null,
"expected_display": "text",
"actual_display": "text",
"verdict": "passed",
"score": 1,
"rationale": "text",
"references": [
1
],
"run_id": "text",
"case_id": "text",
"session_id": "text",
"trace_id": "text"
}
]
}
]
}
],
"by_suite": [
{
"suite_id": "123e4567-e89b-12d3-a456-426614174000",
"suite_name": "text",
"pass_rate": 1,
"passed": 0,
"failed": 0,
"error": 0,
"pending": 0,
"skipped": 0,
"measured": 0,
"latest_run_at": "text",
"cases": [
{
"case_id": "123e4567-e89b-12d3-a456-426614174000",
"case_name": "text",
"pass_rate": 1,
"passed": 0,
"failed": 0,
"error": 0,
"pending": 0,
"skipped": 0,
"measured": 0,
"latest_run_id": "123e4567-e89b-12d3-a456-426614174000",
"latest_run_at": "text",
"assertions": [
{
"eval_key": "text",
"eval_type": "assertion",
"passed": 0,
"failed": 0,
"error": 0,
"pending": 0,
"skipped": 0,
"pass_rate": 1,
"results": [
{
"id": "123e4567-e89b-12d3-a456-426614174000",
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"case_id": "123e4567-e89b-12d3-a456-426614174000",
"session_id": "text",
"eval_key": "text",
"eval_type": "assertion",
"assertion_kind": "text",
"metric_key": "text",
"status": "passed",
"passed": true,
"score": 1,
"expected": null,
"actual": null,
"rationale": "text",
"justification": "text",
"references": [
1
],
"details": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"definition": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"created_at": "text"
}
],
"checks": [
{
"source_type": "metric",
"key": "text",
"label": "text",
"value_type": "numeric",
"evaluation_method": "threshold",
"expected": null,
"actual": null,
"expected_display": "text",
"actual_display": "text",
"verdict": "passed",
"score": 1,
"rationale": "text",
"references": [
1
],
"run_id": "text",
"case_id": "text",
"session_id": "text",
"trace_id": "text"
}
]
}
]
}
]
}
]
}Simulation Bridge
The simulation bridge turns a natural language test objective into a set of autonomous test scenarios and runs them end-to-end against a service.
NL objective to N autonomous scenarios against the tracked simulation primitives.
Returns immediately with run_id + generated scenarios. Scenarios run in the background via BridgeExecutor (lifecycle-managed, shutdown-drained). Completion events are emitted to Delta via world_emitter for SDP observability.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
5203Optional world entity UUID to bind caller context for every scenario session created by this bridge run. The value is forwarded to each scenario's session-create call and inherited by any forks; identical precedence rules apply per-session (entity match wins over phone, stale UUID falls back to phone lookup, malformed UUID returns 422). Use this to pin an entire regression suite to a specific test patient.
falsefalseSuccessful Response
Validation Error
POST /v1/{workspace_id}/simulations/bridge HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 604
{
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"objective": "text",
"num_scenarios": 5,
"max_turns": 20,
"concurrency": 3,
"branch_name": "text",
"entity_id": "123e4567-e89b-12d3-a456-426614174000",
"tags": [
"text"
],
"target_spec": {
"desired_states": [
"text"
],
"non_desired_states": [
{
"state": "text",
"mode": "hard"
}
],
"ordered_pathways": [
{
"id": "text",
"sequence": [
"text"
]
}
],
"completion_criteria": {
"state_based": [
"text"
],
"tool_based": [
"text"
]
}
},
"exploration": {
"coverage_driven_selection": false,
"enable_forking": false,
"max_forks_per_scenario": 2,
"recommend_candidates": 3
},
"auto_target_spec": false,
"async_generation": false
}{
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"status": "running",
"scenarios": [
{
"description": "text",
"temperament": "neutral",
"initial_message": "text",
"scenario_instructions": "text",
"persona": {
"name": "",
"role": "patient",
"communication_style": "",
"default_language": "en-US",
"background": ""
},
"case_id": "123e4567-e89b-12d3-a456-426614174000"
}
],
"case_ids": [
"123e4567-e89b-12d3-a456-426614174000"
],
"inferred_target_spec": {
"desired_states": [
"text"
],
"non_desired_states": [
{
"state": "text",
"mode": "hard"
}
],
"ordered_pathways": [
{
"id": "text",
"sequence": [
"text"
]
}
],
"completion_criteria": {
"state_based": [
"text"
],
"tool_based": [
"text"
]
}
},
"inferred_target_spec_rationale": "text"
}The bridge can infer a target spec when none is supplied, steer scenario generation toward coverage gaps, fork conversations when enabled, and run generation asynchronously. When asynchronous generation is enabled, poll Get Coverage Run for progress.
Returns the created coverage run with generated scenarios. Each scenario includes a persona (name, background, temperament) and conversation strategy (initial message, instructions).
When async_generation is true, the response returns status: running with an empty scenarios array. Use the returned run_id with Get Coverage Run to read generated scenarios and final status as they become available.
When automatic target-spec inference is enabled, the response includes the inferred spec and its rationale.
The bridge response itself carries no execution summary. Read total_sessions and per-session results back through Get Coverage Run once execution finishes.
Each session is tagged with metadata for joining sessions back to their source scenarios: scenario_index:N identifies which generated scenario produced the session, and temperament:X records the persona temperament used.
The bridge creates a tracked coverage run, so results are visible through the existing coverage graph, session list, and scoring endpoints.
This is the REST API equivalent of forge platform sim bridge - same scenario generation and autonomous execution, accessible without the CLI.
Saved Case Benchmarks
Saved-case benchmarks run a tag-selected batch of simulation cases and queue each selected case as its own bridge run. The benchmark response returns immediately with a stable batch_id, the selected case IDs, started run IDs, skipped cases, failed-to-start cases, and an aggregate start summary.
Run a suite or tag-selected benchmark batch as one saved-case run per case.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Optional world entity UUID to bind caller context for every selected saved-case run. When omitted, each case falls back to fixtures.case_specific.entity_id if present.
20201Successful Response
Validation Error
POST /v1/{workspace_id}/simulations/benchmarks/run HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 387
{
"suite_id": "123e4567-e89b-12d3-a456-426614174000",
"required_tags": [
"text"
],
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"entity_id": "123e4567-e89b-12d3-a456-426614174000",
"max_cases": 20,
"max_turns": 20,
"concurrency": 1,
"branch_name": "text",
"tags": [
"text"
],
"exploration": {
"coverage_driven_selection": false,
"enable_forking": false,
"max_forks_per_scenario": 2,
"recommend_candidates": 3
}
}{
"batch_id": "123e4567-e89b-12d3-a456-426614174000",
"suite_id": "123e4567-e89b-12d3-a456-426614174000",
"required_tags": [
"text"
],
"selected_case_ids": [
"123e4567-e89b-12d3-a456-426614174000"
],
"run_ids": [
"123e4567-e89b-12d3-a456-426614174000"
],
"cases": [
{
"case_id": "123e4567-e89b-12d3-a456-426614174000",
"status": "started",
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"result_pointer": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"reason": "text"
}
],
"skipped_cases": [
{
"case_id": "123e4567-e89b-12d3-a456-426614174000",
"status": "started",
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"result_pointer": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"reason": "text"
}
],
"failed_to_start_cases": [
{
"case_id": "123e4567-e89b-12d3-a456-426614174000",
"status": "started",
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"result_pointer": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"reason": "text"
}
],
"aggregate_summary": {
"selected_count": 1,
"started_count": 1,
"skipped_count": 1,
"failed_to_start_count": 1,
"status_counts": {
"ANY_ADDITIONAL_PROPERTY": 1
},
"scored_count": 0,
"average_score": 1
}
}Results
Use the returned run_ids to aggregate completion status, scores, pass counts, metric availability, per-run summaries, and capability breakdowns.
Aggregate benchmark results over the run ids returned by benchmark start.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Successful Response
00000pendingPossible values: 0Validation Error
POST /v1/{workspace_id}/simulations/benchmarks/results HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 52
{
"run_ids": [
"123e4567-e89b-12d3-a456-426614174000"
]
}{
"run_ids": [
"123e4567-e89b-12d3-a456-426614174000"
],
"total_runs": 1,
"status_counts": {
"ANY_ADDITIONAL_PROPERTY": 1
},
"completed_count": 0,
"failed_count": 0,
"scored_count": 0,
"average_score": 1,
"pass_count": 0,
"fail_count": 0,
"metric_status": "pending",
"metric_result_count": 0,
"metrics_last_checked_at": "2026-01-01T00:00:00.000Z",
"per_run": [
{
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"service_id": "123e4567-e89b-12d3-a456-426614174000",
"status": "running",
"error": "text",
"tags": [
"text"
],
"case_ids": [
"123e4567-e89b-12d3-a456-426614174000"
],
"session_ids": [
"text"
],
"total_sessions": 0,
"total_turns": 0,
"terminal_session_count": 0,
"scored_session_count": 0,
"average_score": 1,
"passed": true,
"pass_count": 0,
"fail_count": 0,
"score_rationales": [
"text"
],
"capability_tags": [
"text"
],
"created_at": "text",
"completed_at": "text",
"result_pointer": {
"ANY_ADDITIONAL_PROPERTY": "anything"
},
"immediate_average_score": 1,
"immediate_passed": true,
"immediate_pass_count": 0,
"immediate_fail_count": 0,
"metric_status": "pending",
"metric_result_count": 0,
"metrics_last_checked_at": "2026-01-01T00:00:00.000Z"
}
],
"capability_breakdown": {
"ANY_ADDITIONAL_PROPERTY": {
"run_count": 0,
"status_counts": {
"ANY_ADDITIONAL_PROPERTY": 1
},
"completed_count": 0,
"failed_count": 0,
"scored_count": 0,
"average_score": 1,
"pass_count": 0,
"fail_count": 0
}
},
"missing_run_ids": [
"123e4567-e89b-12d3-a456-426614174000"
]
}Bridge Plan
The separate plan-only bridge endpoint has been removed. Prepare the target specification before submitting it through the supported bridge workflow above. Starting a bridge run executes simulations; it is not a dry-run planning operation.
Write Isolation
Coverage sessions read the workspace's current configuration and run through the production reasoning path, but every tool runs in simulation mode. No tool call reaches an integration, a world write, or a platform function. Reads and writes alike return a simulated result instead of executing:
The mode is fixed when the session starts and applies to direct tool calls, companion-skill sub-tools, and every forked branch. Calls from workspace test caller numbers and text turns from test credentials run tools the same way. The text Playground and browser test calls run tools live, so use them, with test identities and non-production credentials, when a test needs real tool results. Restricted Surface tools remain unavailable in simulations.
This is Tier-1 text simulation: the simulated caller sends text through the production reasoning path, with tools in simulation mode. It does not test telephony, media attachment, speech recognition, text-to-speech, or other audio behavior. Use a real-audio test workflow for those concerns.
Last updated
Was this helpful?