Customer Data Intake

Workspace-scoped dataset ingestion, file versioning, source discovery, processing, and publication, with explicit security and rollout boundaries.

Concurrence's current file-intake workflow uses registered datasets. It gives each upload a named contract, catalog status, and version history, with separate steps for processing and publication.

Scenario	Supported workflow
An operator uploads a CSV snapshot against a defined schema	Authenticated dataset intake
A Google Drive folder should feed a registered dataset	Dataset intake source
A source exposes a stable API that Concurrence should read repeatedly	Connector
A person needs to complete a structured form	Surfaces

The Developer Console exposes dataset intake where the Intake feature is enabled. The former standalone upload-link system and external HMAC upload route have been retired; they are not current alternatives to dataset intake.

Dataset Intake

Dataset intake is the governed path for files that need a named data contract, catalog status, and version history. Authenticated workspace routes let an operator register a dataset, upload or download files, list file status, and refresh a dataset from mapped sources.

Dataset Contracts

Every dataset has one of two ingestion modes:

• Snapshot - A tabular contract with a primary key and typed fields. CSV is the current processing format. Although the registration contract also accepts xls and xlsx type tokens, the current snapshot validators and processor read CSV bytes; use CSV for processing workflows.
• Document - A set of accepted filename extensions plus a document-processing configuration. The current native text extractor supports PDF, DOCX, and UTF-8 text such as Markdown. OCR and hybrid are accepted configuration values but are not current processing implementations and result in a failed processing verdict.

Registering a file type does not prove that a downstream processor can interpret every file with that extension. Validate representative files before production use.

Upload and Version Behavior

An authenticated upload is associated with one registered dataset and is capped at 100 MB. Dataset contracts can record max_size_mb, but the current upload runtime does not enforce a lower per-dataset value; clients that rely on it must enforce it before upload. The service computes a content hash and returns an existing row for a byte-identical file already present in the same workspace and dataset.

Snapshot versions are allocated at the dataset level. Document versions belong to a logical document: omit document_id to create a document, or provide an existing document_id to add a same-type version.

Processing is configuration-dependent:

• Snapshot uploads can be validated synchronously or land as received for asynchronous processing.
• Document uploads land as received and require the document processor to reach a terminal status.
• A successful upload response confirms storage and cataloging, not successful parsing, publication, or world-model projection.

The current intake path does not perform inline malware scanning. An allowlisted extension, MIME type, or successful schema check is not evidence that a file is safe. Workflows that require a malware verdict must arrange that control before submission.

Google Drive Sources

A dataset source maps one or more Google Drive folders to registered datasets. Discovery occurs when an operator triggers a source sync or starts a dataset update; registration alone does not schedule continuous synchronization.

A source sync creates a batch per mapped folder, downloads eligible files, and places newly landed files in the dataset catalog. Unchanged content is deduplicated. Processing a batch is a separate step unless the dataset-update workflow orchestrates it.

The dataset-update endpoint combines source discovery, preparation, and publication behind a durable run. It applies only when an active Drive source maps to the requested dataset. Monitor the returned run rather than assuming that 202 Accepted means the dataset is ready.

Source removal and data erasure are separate. The audited registration-deletion capability in merged source stops future discovery while retaining imported files, batches, published data, upstream files, and source credentials. Its public deployment remains unconfirmed as of September 16, 2026; see Intake Sources before planning source retirement.

Retired Intake Paths

Standalone shareable upload links and the external HMAC upload route have been removed. Existing integration instructions for those paths need migration to authenticated dataset intake or a separately agreed collection workflow. Patient-facing Surfaces remain a distinct form workflow; a form submission does not automatically create a dataset file.

Security and Compliance Boundaries

• Dataset, source, listing, upload, and download operations require a workspace-scoped Platform credential.
• Content-type checks, schema validation, malware scanning, parsing, and downstream publication are distinct controls. Success at one stage does not imply success at another.
• The intake routes do not, by themselves, establish a HIPAA or HITRUST compliance posture. Deployment controls, contractual coverage, retention, residency, and operating procedures remain separate concerns. See Compliance and Audit and Data Residency.

For endpoint-level request and response behavior, see the Platform intake guide.