Customer Data Intake
Register governed intake datasets, upload and download catalog files, refresh mapped Drive sources, and monitor durable dataset processing and update runs.
These endpoints manage the dataset intake catalog: registered dataset contracts, direct file uploads, files discovered by mapped intake sources, and dataset update runs. They require a workspace-scoped Platform bearer credential.
The separate upload-link system has been retired. New file integrations should use the authenticated dataset workflow described here.
Files
List Files
Returns a paginated dataset-file catalog. Results can be sorted by ingestion time, filename, status, dataset, or size; filtered by status; searched by filename; and optionally include withdrawn document versions.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
500-ingested_atfalseSuccessful Response
Validation Error
GET /v1/{workspace_id}/intake/files HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"items": [
{
"id": "123e4567-e89b-12d3-a456-426614174000",
"filename": "text",
"dataset": "text",
"schema_version": "text",
"status": "received",
"error_reason": "text",
"size_bytes": 1,
"ingested_at": "text",
"document_id": "123e4567-e89b-12d3-a456-426614174000",
"version": 1,
"source_type": "text",
"source_file_id": "text",
"source_folder_path": "text",
"doc_metadata": {
"ANY_ADDITIONAL_PROPERTY": "text"
}
}
],
"has_more": true,
"continuation_token": 1,
"total": 1
}The status vocabulary is received, scanned, processing, curated, rejected, failed, and held. Current upload paths commonly use:
received
Stored and waiting for, or eligible for, downstream processing
curated
Validation or configured processing completed successfully
rejected
The file did not satisfy the tabular contract
failed
Document extraction or another processing step failed
scanned, processing, and held are valid wire values, but they do not by themselves prove that malware scanning ran. Use error_reason for failure context and treat unknown future values defensively.
Upload File
Uploads one file as multipart/form-data to a registered dataset. The service enforces a 100 MB request cap.
max_size_mb is stored on the dataset contract and returned by the dataset list, but the current upload runtime does not enforce a lower per-dataset value. Enforce that value in the client if your workflow depends on it.
Before persistence, the service computes a SHA-256 hash and returns the existing file row for byte-identical content already present in the same workspace and dataset. The current upload path does not perform inline malware scanning; arrange a separate pre-submission control if your workflow requires a malware verdict.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
^[a-z0-9][a-z0-9_-]*[a-z0-9]$Successful Response
A file in the Files list (intake-ui-mvp-design.md §5.1).
file_id — used for download.
Schema slug (UI label "Schema").
The contract version this file validated against (e.g. v1).
Null unless rejected/failed.
ISO-8601 timestamp the file was received.
The document this version belongs to (§5.9). Null for snapshot/CSV rows; set for documents — the console groups versions by it and the version chain.
The file's version number (per-document for documents, per-dataset for snapshot). Lets the UI render a document's version history.
Origin of the document this version belongs to — manual for console
uploads, google_shared_drive for the Drive connector (V269). Null for
snapshot/CSV rows.
The source's stable external id (e.g. the Google Drive file id) for the document. Lets the console show provenance and confirm a re-synced file bumped the same document. Null for manual uploads and snapshot rows.
Where the file lived within the mapped Drive folder tree (V279) — the
relative path, folder names joined by / (e.g. clinical/notes); ''
at the mapped root. Lets the console show the original folder structure. Null
for manual uploads and snapshot rows.
Validation Error
POST /v1/{workspace_id}/intake/files HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: multipart/form-data
Accept: */*
Content-Length: 107
{
"dataset": "text",
"file": "text",
"document_id": "123e4567-e89b-12d3-a456-426614174000",
"doc_metadata": "text"
}{
"id": "123e4567-e89b-12d3-a456-426614174000",
"filename": "text",
"dataset": "text",
"schema_version": "text",
"status": "received",
"error_reason": "text",
"size_bytes": 1,
"ingested_at": "text",
"document_id": "123e4567-e89b-12d3-a456-426614174000",
"version": 1,
"source_type": "text",
"source_file_id": "text",
"source_folder_path": "text",
"doc_metadata": {
"ANY_ADDITIONAL_PROPERTY": "text"
}
}Upload behavior depends on the dataset mode:
Snapshot - The current processing format is CSV. Depending on workspace configuration, validation can complete synchronously (
curatedorrejected) or the file can enter asynchronous processing asreceived.Document - The filename extension must be in the dataset's accepted set. Omit
document_idto create a logical document, or pass an existing document ID to add a same-type version. Documents enter asreceivedand require the extraction processor for a terminal verdict.
Common route errors include:
413
File exceeds the global 100 MB cap
422
Missing dataset contract, invalid or empty filename, disallowed document type, invalid document version target
Tag a Document with Its Subject
A document upload can carry doc_metadata, a JSON-encoded form field of string tags that link the document to the subject it is about, for example {"patient_id": "synthetic-123"}. Send 1 to 32 tags. Keys are 1 to 64 characters of letters, digits, _, ., and -; values are 1 to 256 characters. Omit the field for no tags; an empty object is rejected.
Tags are set when the document is created and apply to every later version. Sending
doc_metadatawith an existingdocument_idreturns422, and so does sending it to a snapshot dataset.Byte-identical content deduplicates onto an existing document only when its tags match exactly, so two subjects' identical files stay separate documents.
The Files list returns each document's tags in
doc_metadata({}when untagged,nullfor snapshot files). The upload response echoes the tags sent with that request, so read the Files list for the stored value.
Tags are caller-supplied data. They can hold identifiers about a person, so handle them like the document itself, and do not treat them as verified identity.
Download File
Downloads the original catalog file. The response includes Content-Disposition: attachment and Cache-Control: no-store; a file outside the workspace scope returns 404.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Successful Response
Validation Error
GET /v1/{workspace_id}/intake/files/{file_id}/download HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
No content
Dataset Contracts
List Datasets
Returns registered dataset contracts with their mode, representative file type, accepted document types, schema version, field count, and recorded max_size_mb value.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
500nameSuccessful Response
Validation Error
GET /v1/{workspace_id}/intake/datasets HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"items": [
{
"name": "text",
"schema_version": "text",
"field_count": 1,
"file_type": "text",
"accepted_file_types": [
"text"
],
"ingestion_mode": "text",
"limits": {
"max_size_mb": 1
}
}
],
"has_more": true,
"continuation_token": 1,
"total": 1
}Register a Dataset
The server infers ingestion_mode from file_type:
Snapshot -
csv,xls, orxlsx; requires one or more primary-key columns and fields. The supported field types arestr,int,float,bool,date, anddatetime.Document - Any other lowercase alphanumeric type; ignores the tabular primary key and field list.
accepted_file_typescan contain up to 16 non-tabular types.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Create Schema payload (intake-ui-mvp-design.md §5.4, §5.9).
ingestion_mode is inferred from file_type (csv/xls/xlsx → snapshot;
else document) — not part of the wire contract. Snapshot datasets require
primary_key + schema; document datasets take an optional
document_processing config and ignore primary_key/schema. The backend
applies the other hidden defaults (on_schema_change=additive,
schema_version auto).
^[a-z0-9][a-z0-9_-]*[a-z0-9]$^[a-z0-9]+$Document only — the SET of accepted file types (V265, e.g.
["pdf","docx","md"]). Empty → single-type (just file_type).
file_type is included as the primary. Mixing in a tabular type
(csv/xls/xlsx) is rejected; snapshot datasets are single-type.
[]Snapshot only — the canonical PK column(s). Empty/ignored for documents.
[]Successful Response
A registered schema in the Schemas list (intake-ui-mvp-design.md §5.3).
Dataset slug (UI label "Schema").
Primary/representative file type (the first accepted type).
Document datasets: the full accepted-type set (V265). Empty for snapshot
or legacy single-type documents (the UI falls back to file_type).
[]snapshot (CSV/Excel → CDC) or document (PDF/docx → extraction).
Validation Error
POST /v1/{workspace_id}/intake/schema/register HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 222
{
"name": "text",
"file_type": "text",
"accepted_file_types": [
"text"
],
"primary_key": [
"text"
],
"schema": [
{
"name": "text",
"type": "str"
}
],
"document_processing": {
"retain_source": true,
"extraction_mode": "text_extract"
},
"max_size_mb": 1
}{
"name": "text",
"schema_version": "text",
"field_count": 1,
"file_type": "text",
"accepted_file_types": [
"text"
],
"ingestion_mode": "text",
"limits": {
"max_size_mb": 1
}
}Document contracts accept document_processing.extraction_mode values of text_extract, OCR, or hybrid, but the current processor implements text_extract only. OCR and hybrid reach a failed processing verdict rather than performing OCR.
With text_extract, the current processor handles PDF, DOCX, and UTF-8 text types such as txt, text, md, and markdown. An unknown type is treated as text only when its bytes decode as UTF-8; otherwise processing fails. Registering an accepted extension does not guarantee parser support.
409
A dataset with this name already exists in the workspace
422
Invalid fields, types, primary key, extraction mode, or mixed tabular/document type set
Update a Dataset
POST /v1/{workspace_id}/intake/datasets/{dataset}/update starts the one-click workflow for a dataset backed by at least one active Google Drive source. It checks mapped folders, creates batches for newly landed files, starts preparation, and then starts publication when preparation succeeds.
This route does not refresh a dataset made only from manual uploads. If no active Drive source maps to the dataset, the accepted run transitions to failed with an explanatory error.
Run the customer-facing dataset update flow.
This is the one-click orchestration path for the console: check mapped Drive folders, prepare any newly landed files, and publish immediately when no preparation is needed. If preparation is async, the run remains durable and is advanced by file write-backs or by refresh.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
^[a-z0-9][a-z0-9_-]*[a-z0-9]$Successful Response
Durable status for one dataset update action.
Validation Error
POST /v1/{workspace_id}/intake/datasets/{dataset}/update HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"dataset": "text",
"status": "checking_drive",
"source_ids": [
"123e4567-e89b-12d3-a456-426614174000"
],
"batch_ids": [
"123e4567-e89b-12d3-a456-426614174000"
],
"materialize_run_id": 1,
"error": "text",
"created_at": "text",
"updated_at": "text",
"completed_at": "text"
}The 202 Accepted response is a durable run record, not a completion receipt. Its status is one of checking_drive, preparing, publishing, completed, needs_review, or failed. Poll the run until it reaches a terminal status:
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Successful Response
Durable status for one dataset update action.
Validation Error
GET /v1/{workspace_id}/intake/update-runs/{run_id} HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"run_id": "123e4567-e89b-12d3-a456-426614174000",
"dataset": "text",
"status": "checking_drive",
"source_ids": [
"123e4567-e89b-12d3-a456-426614174000"
],
"batch_ids": [
"123e4567-e89b-12d3-a456-426614174000"
],
"materialize_run_id": 1,
"error": "text",
"created_at": "text",
"updated_at": "text",
"completed_at": "text"
}The run response includes source_ids, batch_ids, timestamps, and optional materialize_run_id and error. Starting another update while one is active can return the existing run rather than create parallel work.
External HMAC Upload
The legacy external HMAC upload route has been removed from the current Platform API. It is no longer a separately provisioned compatibility option. Migrate file integrations to the authenticated dataset upload workflow above; dataset registration and processing remain explicit steps.
Last updated
Was this helpful?